Image coding method and apparatus using motion vector differences

ABSTRACT

According to embodiments of the present document, a prediction procedure can be performed for image video coding, and the prediction procedure can comprise merge mode motion vector differences (MMVD) and symmetric motion vector differences (SMVD) according inter prediction. The inter prediction can be performed on the basis of reference pictures of a current picture, and types of the reference pictures (e.g. a long-term reference picture, a short-term reference picture, etc.) can be considered for the inter prediction. Accordingly, performance and coding efficiency in the prediction procedure can be increased.

This application is a Continuation Application of InternationalApplication No. PCT/KR2020/008138, filed on Jun. 24, 2020, which claimsthe benefit of U.S. Provisional Application No. 62/865,960, filed onJun. 24, 2019, the contents of which are all hereby incorporated byreference herein in their entirety.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The present document is related to an image coding method and apparatususing motion vector differences.

Related Art

Recently, demand for high-resolution, high-quality image/video such as4K or 8K or higher ultra high definition (UHD) image/video has increasedin various fields. As image/video data has high resolution and highquality, the amount of information or bits to be transmitted increasesrelative to the existing image/video data, and thus, transmitting imagedata using a medium such as an existing wired/wireless broadband line oran existing storage medium or storing image/video data using existingstorage medium increase transmission cost and storage cost.

In addition, interest and demand for immersive media such as virtualreality (VR) and artificial reality (AR) content or holograms hasrecently increased and broadcasting for image/video is havingcharacteristics different from reality images such as game images hasincreased.

Accordingly, a highly efficient image/video compression technology isrequired to effectively compress, transmit, store, and reproduceinformation of a high-resolution, high-quality image/video havingvarious characteristics as described above.

In particular, inter prediction in image/video coding may use motionvector differences. There is a discussion of deriving motion vectordifferences based on reference picture types (for example, short-term orlong-term reference picture) with respect to the procedures.

SUMMARY

According to an embodiment of the present document, a method and adevice for improving image/video coding efficiency are provided.

According to an embodiment of the present document, a method and adevice for performing inter prediction efficiently in an image/videocoding system are provided.

According to an embodiment of the present document, a method and adevice for signaling information on motion vector differences for interprediction are provided.

According to an embodiment of the present document, a method and adevice for signaling information on L0 motion vector differences and L1motion vector differences are provided when bi-prediction is applied toa current block.

According to an embodiment of the present document, a method and adevice for signaling an SMVD flag are provided.

According to an embodiment of the present disclosure, a specificreference picture type may be used for deriving symmetric motion vectordifferences.

According to an embodiment of the present document, a procedure forderiving SMVD reference indices using short-term reference pictures(pictures marked as being used for short-term reference).

According to an embodiment of the present document, a video/imagedecoding method performed by a decoding apparatus is provided.

According to an embodiment of the present document, a decoding apparatusfor performing video/image decoding is provided.

According to an embodiment of the present document, a video/imageencoding method performed by an encoding apparatus is provided.

According to an embodiment of the present document, an encodingapparatus for performing video/image encoding is provided.

According to one embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded video/imageinformation, generated according to the video/image encoding methoddisclosed in at least one of the embodiments of the present document, isstored.

According to an embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded information orencoded video/image information, causing to perform the video/imagedecoding method disclosed in at least one of the embodiments of thepresent document by the decoding apparatus, is stored.

Advantageous Effects

According to the present disclosure, the overall image/video compressionefficiency may be improved.

According to the present disclosure, signaling of information on motionvector differences may be performed efficiently.

According to the present disclosure, L1 motion vector differences may beefficiently derived when bi-prediction is applied to a current block.

According to the present disclosure, information used for deriving L1motion vector differences may be signaled based on the types ofreference pictures, and thus the complexity of a coding system may bereduced.

According to one embodiment of the present disclosure, efficient interprediction may be performed using a specific reference picture type forderiving a reference picture index of SMVD.

The technical effects achieved through specific embodiments of thepresent disclosure are not limited to those described above. Forexample, various other technical effects may be obtained, which may beunderstood or derived from the present disclosure by a person havingordinary skills in the related art. Therefore, specific effects of thepresent disclosure are not limited to the embodiments disclosedexplicitly in this document and may include various other effects thatmay be understood or derived from the technical characteristics of thepresent disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

FIG. 4 shows an example of a video/image encoding method based on interprediction.

FIG. 5 shows an example of a video/image decoding method based on interprediction.

FIG. 6 exemplarily shows an inter prediction procedure.

FIG. 7 is a diagram for describing symmetric motion vector differences(SMVD).

FIG. 8 is a diagram for describing a method of deriving motion vectorsin inter prediction.

FIGS. 9 to 13 illustrate MVD derivation methods of MMVD according to theembodiments of the present disclosure.

FIG. 14 is a diagram for describing SMVD according to one embodiment ofthe present disclosure.

FIG. 15 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure.

FIG. 16 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure.

FIG. 17 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure.

FIGS. 18 and 19 illustrate a video/image encoding method and one exampleof a related component according to an embodiment(s) of the presentdisclosure.

FIGS. 20 and 21 illustrate an image/video decoding method and oneexample of a related component according to an embodiment(s) of thepresent disclosure.

FIG. 22 illustrates an example of a content streaming system to whichthe embodiments of the present disclosure may be applied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present document may be modified in various forms, and specificembodiments thereof are described and shown in the drawings. However,the embodiments are not intended for limiting the present document. Theterms used in the following description are used to merely describespecific embodiments, but are not intended to limit the presentdocument. An expression of a singular number includes an expression ofthe plural number, so long as it is clearly read differently. The termssuch as “include” and “have” are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and it should be thusunderstood that the possibility of existence or addition of one or moredifferent features, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

Meanwhile, each configuration in the drawings described in the presentdocument is shown independently for the convenience of descriptionregarding different characteristic functions, and does not mean thateach configuration is implemented as separate hardware or separatesoftware. For example, two or more components among each component maybe combined to form one component, or one component may be divided intoa plurality of components. Embodiments in which each component isintegrated and/or separated are also included in the scope of thedisclosure of the present document.

Hereinafter, examples of the present embodiment are described in detailwith reference to the accompanying drawings. In addition, like referencenumerals are used to indicate like elements throughout the drawings, andthe same descriptions on the like elements are omitted.

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

Referring to FIG. 1, a video/image coding system may include a firstdevice (a source device) and a second device (a reception device). Thesource device may transmit encoded video/image information or data tothe reception device through a digital storage medium or network in theform of a file or streaming.

The source device may include a video source, an encoding apparatus, anda transmitter. The receiving device may include a receiver, a decodingapparatus, and a renderer. The encoding apparatus may be called avideo/image encoding apparatus, and the decoding apparatus may be calleda video/image decoding apparatus. The transmitter may be included in theencoding apparatus. The receiver may be included in the decodingapparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.

The video source may acquire video/image through a process of capturing,synthesizing, or generating the video/image. The video source mayinclude a video/image capture device and/or a video/image generatingdevice. The video/image capture device may include, for example, one ormore cameras, video/image archives including previously capturedvideo/images, and the like. The video/image generating device mayinclude, for example, computers, tablets and smartphones, and may(electronically) generate video/images. For example, a virtualvideo/image may be generated through a computer or the like. In thiscase, the video/image capturing process may be replaced by a process ofgenerating related data.

The encoding apparatus may encode input video/image. The encodingapparatus may perform a series of procedures such as prediction,transform, and quantization for compaction and coding efficiency. Theencoded data (encoded video/image information) may be output in the formof a bitstream.

The transmitter may transmit the encoded image/image information or dataoutput in the form of a bitstream to the receiver of the receivingdevice through a digital storage medium or a network in the form of afile or streaming. The digital storage medium may include variousstorage mediums such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and thelike. The transmitter may include an element for generating a media filethrough a predetermined file format and may include an element fortransmission through a broadcast/communication network. The receiver mayreceive/extract the bitstream and transmit the received bitstream to thedecoding apparatus.

The decoding apparatus may decode the video/image by performing a seriesof procedures such as dequantization, inverse transform, and predictioncorresponding to the operation of the encoding apparatus.

The renderer may render the decoded video/image. The renderedvideo/image may be displayed through the display.

The present document relates to video/image coding. For example, amethod/embodiment disclosed in the present document may be applied to amethod disclosed in the versatile video coding (VVC) standard, theessential video coding (EVC) standard, the AOMedia Video 1 (AV1)standard, the 2nd generation of audio video coding standard (AVS2) orthe next generation video/image coding standard (e.g., H.267, H.268, orthe like).

The present document suggests various embodiments of video/image coding,and the above embodiments may also be performed in combination with eachother unless otherwise specified.

In the present document, a video may refer to a series of images overtime. A picture generally refers to the unit representing one image at aparticular time frame, and a slice/tile refers to the unit constitutinga part of the picture in terms of coding. A slice/tile may include oneor more coding tree units (CTUs). One picture may consist of one or moreslices/tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles. A brick may represent arectangular region of CTU rows within a tile in a picture. A tile may bepartitioned into a multiple bricks, each of which may be constructedwith one or more CTU rows within the tile. A tile that is notpartitioned into multiple bricks may also be referred to as a brick. Abrick scan may represent a specific sequential ordering of CTUspartitioning a picture, wherein the CTUs may be ordered in a CTU rasterscan within a brick, and bricks within a tile may be orderedconsecutively in a raster scan of the bricks of the tile, and tiles in apicture may be ordered consecutively in a raster scan of the tiles ofthe picture. A tile is a rectangular region of CTUs within a particulartile column and a particular tile row in a picture. The tile column is arectangular region of CTUs having a height equal to the height of thepicture and a width specified by syntax elements in the pictureparameter set. The tile row is a rectangular region of CTUs having aheight specified by syntax elements in the picture parameter set and awidth equal to the width of the picture. A tile scan is a specificsequential ordering of CTUs partitioning a picture in which the CTUs areordered consecutively in CTU raster scan in a tile whereas tiles in apicture are ordered consecutively in a raster scan of the tiles of thepicture. A slice includes an integer number of bricks of a picture thatmay be exclusively contained in a single NAL unit. A slice may consistof either a number of complete tiles or only a consecutive sequence ofcomplete bricks of one tile. In the present document, a tile group and aslice may be used in place of each other. For example, in the presentdocument, a tile group/tile group header may be referred to as aslice/slice header.

Meanwhile, one picture may be divided into two or more subpictures. Asubpicture may be a rectangular region of one or more slices within apicture.

A pixel or a pel may mean a smallest unit constituting one picture (orimage). Also, ‘sample’ may be used as a term corresponding to a pixel. Asample may generally represent a pixel or a value of a pixel, and mayrepresent only a pixel/pixel value of a luma component or only apixel/pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit mayinclude at least one of a specific region of the picture and informationrelated to the region. One unit may include one luma block and twochroma (ex. cb, cr) blocks. The unit may be used interchangeably withterms such as block or area in some cases. In a general case, an M×Nblock may include samples (or sample arrays) or a set (or array) oftransform coefficients of M columns and N rows. Alternatively, thesample may mean a pixel value in the spatial domain, and when such apixel value is transformed to the frequency domain, it may mean atransform coefficient in the frequency domain.

In the present document, “A or B” may mean “only A”, “only B” or “both Aand B”. In other words, “A or B” in the present document may beinterpreted as “A and/or B”. For example, in the present document “A, Bor C (A, B or C)” means “only A”, “only B”, “only C”, or “anycombination of A, B and C”.

A slash (/) or comma (comma) used in the present document may mean“and/or”. For example, “A/B” may mean “A and/or B”. Accordingly, “A/B”may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C”may mean “A, B, or C”.

In the present document, “at least one of A and B” may mean “only A”,“only B” or “both A and B”. Also, in the present document, theexpression “at least one of A or B” or “at least one of A and/or B” maybe interpreted the same as “at least one of A and B”.

Also, in the present document, “at least one of A, B and C” means “onlyA”, “only B”, “only C”, or “any combination of A, B and C”. Also, “atleast one of A, B or C” or “at least one of A, B and/or C” may mean “atleast one of A, B and C”.

Also, parentheses used in the present document may mean “for example”.Specifically, when “prediction (intra prediction)” is indicated, “intraprediction” may be proposed as an example of “prediction”. In otherwords, “prediction” in the present document is not limited to “intraprediction”, and “intra prediction” may be proposed as an example of“prediction”. Also, even when “prediction (i.e., intra prediction)” isindicated, “intra prediction” may be proposed as an example of“prediction”.

Technical features that are individually described in one drawing in thepresent document may be implemented individually or simultaneously.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied. Hereinafter, what is referred to as the videoencoding apparatus may include an image encoding apparatus.

Referring to FIG. 2, the encoding apparatus 200 includes an imagepartitioner 210, a predictor 220, a residual processor 230, and anentropy encoder 240, an adder 250, a filter 260, and a memory 270. Thepredictor 220 may include an inter predictor 221 and an intra predictor222. The residual processor 230 may include a transformer 232, aquantizer 233, a dequantizer 234, and an inverse transformer 235. Theresidual processor 230 may further include a subtractor 231. The adder250 may be called a reconstructor or a reconstructed block generator.The image partitioner 210, the predictor 220, the residual processor230, the entropy encoder 240, the adder 250, and the filter 260 may beconfigured by at least one hardware component (ex. An encoder chipset orprocessor) according to an embodiment. In addition, the memory 270 mayinclude a decoded picture buffer (DPB) or may be configured by a digitalstorage medium. The hardware component may further include the memory270 as an internal/external component.

The image partitioner 210 may partition an input image (or a picture ora frame) input to the encoding apparatus 200 into one or moreprocessors. For example, the processor may be called a coding unit (CU).In this case, the coding unit may be recursively partitioned accordingto a quad-tree binary-tree ternary-tree (QTBTTT) structure from a codingtree unit (CTU) or a largest coding unit (LCU). For example, one codingunit may be partitioned into a plurality of coding units of a deeperdepth based on a quad tree structure, a binary tree structure, and/or aternary structure. In this case, for example, the quad tree structuremay be applied first and the binary tree structure and/or ternarystructure may be applied later. Alternatively, the binary tree structuremay be applied first. The coding procedure according to the presentdisclosure may be performed based on the final coding unit that is nolonger partitioned. In this case, the largest coding unit may be used asthe final coding unit based on coding efficiency according to imagecharacteristics, or if necessary, the coding unit may be recursivelypartitioned into coding units of deeper depth and a coding unit havingan optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure of prediction, transform, andreconstruction, which are described later. As another example, theprocessor may further include a prediction unit (PU) or a transform unit(TU). In this case, the prediction unit and the transform unit may besplit or partitioned from the aforementioned final coding unit. Theprediction unit may be a unit of sample prediction, and the transformunit may be a unit for deriving a transform coefficient and/or a unitfor deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area insome cases. In a general case, an M×N block may represent a set ofsamples or transform coefficients composed of M columns and N rows. Asample may generally represent a pixel or a value of a pixel, mayrepresent only a pixel/pixel value of a luma component or represent onlya pixel/pixel value of a chroma component. A sample may be used as aterm corresponding to one picture (or image) for a pixel or a pel.

In the encoding apparatus 200, a prediction signal (predicted block,prediction sample array) output from the inter predictor 221 or theintra predictor 222 is subtracted from an input image signal (originalblock, original sample array) to generate a residual signal residualblock, residual sample array), and the generated residual signal istransmitted to the transformer 232. In this case, as shown, a unit forsubtracting a prediction signal (predicted block, prediction samplearray) from the input image signal (original block, original samplearray) in the encoder 200 may be called a subtractor 231. The predictormay perform prediction on a block to be processed (hereinafter, referredto as a current block) and generate a predicted block includingprediction samples for the current block. The predictor may determinewhether intra prediction or inter prediction is applied on a currentblock or CU basis. As described later in the description of eachprediction mode, the predictor may generate various information relatedto prediction, such as prediction mode information, and transmit thegenerated information to the entropy encoder 240. The information on theprediction may be encoded in the entropy encoder 240 and output in theform of a bitstream.

The intra predictor 222 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The non-directional mode may include, for example,a DC mode and a planar mode. The directional mode may include, forexample, 33 directional prediction modes or 65 directional predictionmodes according to the degree of detail of the prediction direction.However, this is merely an example, more or less directional predictionmodes may be used depending on a setting. The intra predictor 222 maydetermine the prediction mode applied to the current block by using aprediction mode applied to a neighboring block.

The inter predictor 221 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. Here, in order to reduce theamount of motion information transmitted in the inter prediction mode,the motion information may be predicted in units of blocks, sub-blocks,or samples based on correlation of motion information between theneighboring block and the current block. The motion information mayinclude a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0prediction, L1 prediction, Bi prediction, etc.) information. In the caseof inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. The referencepicture including the reference block and the reference pictureincluding the temporal neighboring block may be the same or different.The temporal neighboring block may be called a collocated referenceblock, a co-located CU (colCU), and the like, and the reference pictureincluding the temporal neighboring block may be called a collocatedpicture (colPic). For example, the inter predictor 221 may configure amotion information candidate list based on neighboring blocks andgenerate information indicating which candidate is used to derive amotion vector and/or a reference picture index of the current block.Inter prediction may be performed based on various prediction modes. Forexample, in the case of a skip mode and a merge mode, the interpredictor 221 may use motion information of the neighboring block asmotion information of the current block. In the skip mode, unlike themerge mode, the residual signal may not be transmitted. In the case ofthe motion vector prediction (MVP) mode, the motion vector of theneighboring block may be used as a motion vector predictor and themotion vector of the current block may be indicated by signaling amotion vector difference.

The predictor 220 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply both intra prediction and inter prediction.This may be called combined inter and intra prediction (CIIP). Inaddition, the predictor may be based on an intra block copy (IBC)prediction mode or a palette mode for prediction of a block. The IBCprediction mode or palette mode may be used for content image/videocoding of a game or the like, for example, screen content coding (SCC).The IBC basically performs prediction in the current picture but may beperformed similarly to inter prediction in that a reference block isderived in the current picture. That is, the IBC may use at least one ofthe inter prediction techniques described in the present disclosure. Thepalette mode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The prediction signal generated by the predictor (including the interpredictor 221 and/or the intra predictor 222) may be used to generate areconstructed signal or to generate a residual signal. The transformer232 may generate transform coefficients by applying a transformtechnique to the residual signal. For example, the transform techniquemay include at least one of a discrete cosine transform (DCT), adiscrete sine transform (DST), a karhunen-loève transform (KLT), agraph-based transform (GBT), or a conditionally non-linear transform(CNT). Here, the GBT means transform obtained from a graph whenrelationship information between pixels is represented by the graph. TheCNT refers to transform generated based on a prediction signal generatedusing all previously reconstructed pixels. In addition, the transformprocess may be applied to square pixel blocks having the same size ormay be applied to blocks having a variable size rather than square.

The quantizer 233 may quantize the transform coefficients and transmitthem to the entropy encoder 240 and the entropy encoder 240 may encodethe quantized signal (information on the quantized transformcoefficients) and output a bitstream. The information on the quantizedtransform coefficients may be referred to as residual information. Thequantizer 233 may rearrange block type quantized transform coefficientsinto a one-dimensional vector form based on a coefficient scanning orderand generate information on the quantized transform coefficients basedon the quantized transform coefficients in the one-dimensional vectorform. Information on transform coefficients may be generated. Theentropy encoder 240 may perform various encoding methods such as, forexample, exponential Golomb, context-adaptive variable length coding(CAVLC), context-adaptive binary arithmetic coding (CABAC), and thelike. The entropy encoder 240 may encode information necessary forvideo/image reconstruction other than quantized transform coefficients(ex. values of syntax elements, etc.) together or separately. Encodedinformation (ex. encoded video/image information) may be transmitted orstored in units of NALs (network abstraction layer) in the form of abitstream. The video/image information may further include informationon various parameter sets such as an adaptation parameter set (APS), apicture parameter set (PPS), a sequence parameter set (SPS), or a videoparameter set (VPS). In addition, the video/image information mayfurther include general constraint information. In the presentdisclosure, information and/or syntax elements transmitted/signaled fromthe encoding apparatus to the decoding apparatus may be included invideo/picture information. The video/image information may be encodedthrough the above-described encoding procedure and included in thebitstream. The bitstream may be transmitted over a network or may bestored in a digital storage medium. The network may include abroadcasting network and/or a communication network, and the digitalstorage medium may include various storage media such as USB, SD, CD,DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown)transmitting a signal output from the entropy encoder 240 and/or astorage unit (not shown) storing the signal may be included asinternal/external element of the encoding apparatus 200, andalternatively, the transmitter may be included in the entropy encoder240.

The quantized transform coefficients output from the quantizer 233 maybe used to generate a prediction signal. For example, the residualsignal (residual block or residual samples) may be reconstructed byapplying dequantization and inverse transform to the quantized transformcoefficients through the dequantizer 234 and the inverse transformer235. The adder 250 adds the reconstructed residual signal to theprediction signal output from the inter predictor 221 or the intrapredictor 222 to generate a reconstructed signal (reconstructed picture,reconstructed block, reconstructed sample array). If there is noresidual for the block to be processed, such as a case where the skipmode is applied, the predicted block may be used as the reconstructedblock. The adder 250 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used forintra prediction of a next block to be processed in the current pictureand may be used for inter prediction of a next picture through filteringas described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied duringpicture encoding and/or reconstruction.

The filter 260 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter260 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 270, specifically, a DPB of thememory 270. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like. The filter 260 may generate variousinformation related to the filtering and transmit the generatedinformation to the entropy encoder 240 as described later in thedescription of each filtering method. The information related to thefiltering may be encoded by the entropy encoder 240 and output in theform of a bitstream.

The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 221. When the interprediction is applied through the encoding apparatus, predictionmismatch between the encoding apparatus 200 and the decoding apparatus300 may be avoided and encoding efficiency may be improved.

The DPB of the memory 270 DPB may store the modified reconstructedpicture for use as a reference picture in the inter predictor 221. Thememory 270 may store the motion information of the block from which themotion information in the current picture is derived (or encoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 221 and used as the motion information of thespatial neighboring block or the motion information of the temporalneighboring block. The memory 270 may store reconstructed samples ofreconstructed blocks in the current picture and may transfer thereconstructed samples to the intra predictor 222.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

Referring to FIG. 3, the decoding apparatus 300 may include an entropydecoder 310, a residual processor 320, a predictor 330, an adder 340, afilter 350, a memory 360. The predictor 330 may include an interpredictor 331 and an intra predictor 332. The residual processor 320 mayinclude a dequantizer 321 and an inverse transformer 321. The entropydecoder 310, the residual processor 320, the predictor 330, the adder340, and the filter 350 may be configured by a hardware component (ex. Adecoder chipset or a processor) according to an embodiment. In addition,the memory 360 may include a decoded picture buffer (DPB) or may beconfigured by a digital storage medium. The hardware component mayfurther include the memory 360 as an internal/external component.

When a bitstream including video/image information is input, thedecoding apparatus 300 may reconstruct an image corresponding to aprocess in which the video/image information is processed in theencoding apparatus of FIG. 2. For example, the decoding apparatus 300may derive units/blocks based on block partition related informationobtained from the bitstream. The decoding apparatus 300 may performdecoding using a processor applied in the encoding apparatus. Thus, theprocessor of decoding may be a coding unit, for example, and the codingunit may be partitioned according to a quad tree structure, binary treestructure and/or ternary tree structure from the coding tree unit or thelargest coding unit. One or more transform units may be derived from thecoding unit. The reconstructed image signal decoded and output throughthe decoding apparatus 300 may be reproduced through a reproducingapparatus.

The decoding apparatus 300 may receive a signal output from the encodingapparatus of FIG. 2 in the form of a bitstream, and the received signalmay be decoded through the entropy decoder 310. For example, the entropydecoder 310 may parse the bitstream to derive information (ex.video/image information) necessary for image reconstruction (or picturereconstruction). The video/image information may further includeinformation on various parameter sets such as an adaptation parameterset (APS), a picture parameter set (PPS), a sequence parameter set(SPS), or a video parameter set (VPS). In addition, the video/imageinformation may further include general constraint information. Thedecoding apparatus may further decode picture based on the informationon the parameter set and/or the general constraint information.Signaled/received information and/or syntax elements described later inthe present disclosure may be decoded may decode the decoding procedureand obtained from the bitstream. For example, the entropy decoder 310decodes the information in the bitstream based on a coding method suchas exponential Golomb coding, CAVLC, or CABAC, and output syntaxelements required for image reconstruction and quantized values oftransform coefficients for residual. More specifically, the CABACentropy decoding method may receive a bin corresponding to each syntaxelement in the bitstream, determine a context model using a decodingtarget syntax element information, decoding information of a decodingtarget block or information of a symbol/bin decoded in a previous stage,and perform an arithmetic decoding on the bin by predicting aprobability of occurrence of a bin according to the determined contextmodel, and generate a symbol corresponding to the value of each syntaxelement. In this case, the CABAC entropy decoding method may update thecontext model by using the information of the decoded symbol/bin for acontext model of a next symbol/bin after determining the context model.The information related to the prediction among the information decodedby the entropy decoder 310 may be provided to the predictor (the interpredictor 332 and the intra predictor 331), and the residual value onwhich the entropy decoding was performed in the entropy decoder 310,that is, the quantized transform coefficients and related parameterinformation, may be input to the residual processor 320. The residualprocessor 320 may derive the residual signal (the residual block, theresidual samples, the residual sample array). In addition, informationon filtering among information decoded by the entropy decoder 310 may beprovided to the filter 350. Meanwhile, a receiver (not shown) forreceiving a signal output from the encoding apparatus may be furtherconfigured as an internal/external element of the decoding apparatus300, or the receiver may be a component of the entropy decoder 310.Meanwhile, the decoding apparatus according to the present disclosuremay be referred to as a video/image/picture decoding apparatus, and thedecoding apparatus may be classified into an information decoder(video/image/picture information decoder) and a sample decoder(video/image/picture sample decoder). The information decoder mayinclude the entropy decoder 310, and the sample decoder may include atleast one of the dequantizer 321, the inverse transformer 322, the adder340, the filter 350, the memory 360, the inter predictor 332, and theintra predictor 331.

The dequantizer 321 may dequantize the quantized transform coefficientsand output the transform coefficients. The dequantizer 321 may rearrangethe quantized transform coefficients in the form of a two-dimensionalblock form. In this case, the rearrangement may be performed based onthe coefficient scanning order performed in the encoding apparatus. Thedequantizer 321 may perform dequantization on the quantized transformcoefficients by using a quantization parameter (ex. quantization stepsize information) and obtain transform coefficients.

The inverse transformer 322 inversely transforms the transformcoefficients to obtain a residual signal (residual block, residualsample array).

The predictor may perform prediction on the current block and generate apredicted block including prediction samples for the current block. Thepredictor may determine whether intra prediction or inter prediction isapplied to the current block based on the information on the predictionoutput from the entropy decoder 310 and may determine a specificintra/inter prediction mode.

The predictor 320 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply intra prediction and inter prediction. Thismay be called combined inter and intra prediction (CIIP). In addition,the predictor may be based on an intra block copy (IBC) prediction modeor a palette mode for prediction of a block. The IBC prediction mode orpalette mode may be used for content image/video coding of a game or thelike, for example, screen content coding (SCC). The IBC basicallyperforms prediction in the current picture but may be performedsimilarly to inter prediction in that a reference block is derived inthe current picture. That is, the IBC may use at least one of the interprediction techniques described in the present disclosure. The palettemode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The intra predictor 331 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The intra predictor 331 may determine theprediction mode applied to the current block by using a prediction modeapplied to a neighboring block.

The inter predictor 332 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. In this case, in order to reducethe amount of motion information transmitted in the inter predictionmode, motion information may be predicted in units of blocks,sub-blocks, or samples based on correlation of motion informationbetween the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index.The motion information may further include inter prediction direction(L0 prediction, L1 prediction, Bi prediction, etc.) information. In thecase of inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. For example, theinter predictor 332 may configure a motion information candidate listbased on neighboring blocks and derive a motion vector of the currentblock and/or a reference picture index based on the received candidateselection information. Inter prediction may be performed based onvarious prediction modes, and the information on the prediction mayinclude information indicating a mode of inter prediction for thecurrent block.

The adder 340 may generate a reconstructed signal (reconstructedpicture, reconstructed block, reconstructed sample array) by adding theobtained residual signal to the prediction signal (predicted block,predicted sample array) output from the predictor (including the interpredictor 332 and/or the intra predictor 331). If there is no residualfor the block to be processed, such as when the skip mode is applied,the predicted block may be used as the reconstructed block.

The adder 340 may be called reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for intraprediction of a next block to be processed in the current picture, maybe output through filtering as described below, or may be used for interprediction of a next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in thepicture decoding process.

The filter 350 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter350 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 360, specifically, a DPB of thememory 360. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 360may be used as a reference picture in the inter predictor 332. Thememory 360 may store the motion information of the block from which themotion information in the current picture is derived (or decoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 260 so as to be utilized as the motion informationof the spatial neighboring block or the motion information of thetemporal neighboring block. The memory 360 may store reconstructedsamples of reconstructed blocks in the current picture and transfer thereconstructed samples to the intra predictor 331.

In the present document, the embodiments described in the filter 260,the inter predictor 221, and the intra predictor 222 of the encodingapparatus 200 may be the same as or respectively applied to correspondto the filter 350, the inter predictor 332, and the intra predictor 331of the decoding apparatus 300. The same may also apply to the unit 332and the intra predictor 331.

As described above, in video coding, prediction is performed to increasecompression efficiency. Through this, it is possible to generate apredicted block including prediction samples for a current block, whichis a block to be coded. Here, the predicted block includes predictionsamples in a spatial domain (or pixel domain). The predicted block isderived equally from the encoding device and the decoding device, andthe encoding device decodes information (residual information) on theresidual between the original block and the predicted block, not theoriginal sample value of the original block itself. By signaling to thedevice, image coding efficiency can be increased. The decoding apparatusmay derive a residual block including residual samples based on theresidual information, and generate a reconstructed block includingreconstructed samples by summing the residual block and the predictedblock, and generate a reconstructed picture including reconstructedblocks.

The residual information may be generated through transformation andquantization processes. For example, the encoding apparatus may derive aresidual block between the original block and the predicted block, andperform a transform process on residual samples (residual sample array)included in the residual block to derive transform coefficients, andthen, by performing a quantization process on the transformcoefficients, derive quantized transform coefficients to signal theresidual related information to the decoding apparatus (via abitstream). Here, the residual information may include locationinformation, a transform technique, a transform kernel, and aquantization parameter, value information of the quantized transformcoefficients etc. The decoding apparatus may performdequantization/inverse transformation process based on the residualinformation and derive residual samples (or residual blocks). Thedecoding apparatus may generate a reconstructed picture based on thepredicted block and the residual block. The encoding apparatus may alsodequantize/inverse transform the quantized transform coefficients forreference for inter prediction of a later picture to derive a residualblock, and generate a reconstructed picture based thereon.

In the present document, at least one of quantization/dequantizationand/or transform/inverse transform may be omitted. When thequantization/dequantization is omitted, the quantized transformcoefficient may be referred to as a transform coefficient. When thetransform/inverse transform is omitted, the transform coefficients maybe called coefficients or residual coefficients, or may still be calledtransform coefficients for uniformity of expression.

In the present document, a quantized transform coefficient and atransform coefficient may be referred to as a transform coefficient anda scaled transform coefficient, respectively. In this case, the residualinformation may include information on transform coefficient(s), and theinformation on the transform coefficient(s) may be signaled throughresidual coding syntax. Transform coefficients may be derived based onthe residual information (or information on the transformcoefficient(s)), and scaled transform coefficients may be derivedthrough inverse transform (scaling) on the transform coefficients.Residual samples may be derived based on an inverse transform(transform) of the scaled transform coefficients. This may beapplied/expressed in other parts of the present document as well.

Intra prediction may refer to prediction that generates predictionsamples for the current block based on reference samples in a picture towhich the current block belongs (hereinafter, referred to as a currentpicture). When intra prediction is applied to the current block,neighboring reference samples to be used for intra prediction of thecurrent block may be derived. The neighboring reference samples of thecurrent block may include samples adjacent to the left boundary of thecurrent block having a size of nW×nH and a total of 2×nH samplesneighboring the bottom-left, samples adjacent to the top boundary of thecurrent block and a total of 2×nW samples neighboring the top-right, andone sample neighboring the top-left of the current block. Alternatively,the neighboring reference samples of the current block may include aplurality of upper neighboring samples and a plurality of leftneighboring samples. In addition, the neighboring reference samples ofthe current block may include a total of nH samples adjacent to theright boundary of the current block having a size of nW×nH, a total ofnW samples adjacent to the bottom boundary of the current block, and onesample neighboring (bottom-right) neighboring bottom-right of thecurrent block.

However, some of the neighboring reference samples of the current blockmay not be decoded yet or available. In this case, the decoder mayconfigure the neighboring reference samples to use for prediction bysubstituting the samples that are not available with the availablesamples. Alternatively, neighboring reference samples to be used forprediction may be configured through interpolation of the availablesamples.

When the neighboring reference samples are derived, (i) the predictionsample may be derived based on the average or interpolation ofneighboring reference samples of the current block, and (ii) theprediction sample may be derived based on the reference sample presentin a specific (prediction) direction for the prediction sample among theperiphery reference samples of the current block. The case of (i) may becalled non-directional mode or non-angular mode and the case of (ii) maybe called directional mode or angular mode.

Furthermore, the prediction sample may also be generated throughinterpolation between the second neighboring sample and the firstneighboring sample located in a direction opposite to the predictiondirection of the intra prediction mode of the current block based on theprediction sample of the current block among the neighboring referencesamples. The above case may be referred to as linear interpolation intraprediction (LIP). In addition, chroma prediction samples may begenerated based on luma samples using a linear model. This case may becalled LM mode.

In addition, a temporary prediction sample of the current block may bederived based on filtered neighboring reference samples, and at leastone reference sample derived according to the intra prediction modeamong the existing neighboring reference samples, that is, unfilteredneighboring reference samples, and the temporary prediction sample maybe weighted-summed to derive the prediction sample of the current block.The above case may be referred to as position dependent intra prediction(PDPC).

In addition, a reference sample line having the highest predictionaccuracy among the neighboring multi-reference sample lines of thecurrent block may be selected to derive the prediction sample by usingthe reference sample located in the prediction direction on thecorresponding line, and then the reference sample line used herein maybe indicated (signaled) to the decoding apparatus, thereby performingintra-prediction encoding. The above case may be referred to asmulti-reference line (MRL) intra prediction or MRL based intraprediction.

In addition, intra prediction may be performed based on the same intraprediction mode by dividing the current block into vertical orhorizontal subpartitions, and neighboring reference samples may bederived and used in the subpartition unit. That is, in this case, theintra prediction mode for the current block is equally applied to thesubpartitions, and the intra prediction performance may be improved insome cases by deriving and using the neighboring reference samples inthe subpartition unit. Such a prediction method may be called intrasub-partitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intraprediction type separately from the intra prediction mode. The intraprediction type may be called in various terms such as an intraprediction technique or an additional intra prediction mode. Forexample, the intra prediction type (or additional intra prediction mode)may include at least one of the above-described LIP, PDPC, MRL, and ISP.A general intra prediction method except for the specific intraprediction type such as LIP, PDPC, MRL, or ISP may be called a normalintra prediction type. The normal intra prediction type may be generallyapplied when the specific intra prediction type is not applied, andprediction may be performed based on the intra prediction mode describedabove. Meanwhile, post-filtering may be performed on the predictedsample derived as needed.

Specifically, the intra prediction procedure may include an intraprediction mode/type determination step, a neighboring reference samplederivation step, and an intra prediction mode/type based predictionsample derivation step. In addition, a post-filtering step may beperformed on the predicted sample derived as needed.

When intra prediction is applied, the intra prediction mode applied tothe current block may be determined using the intra prediction mode ofthe neighboring block. For example, the decoding apparatus may selectone of most probable mode (mpm) candidates of an mpm list derived basedon the intra prediction mode of the neighboring block (ex. left and/orupper neighboring blocks) of the current block based on the received mpmindex and select one of the other remaining intro prediction modes notincluded in the mpm candidates (and planar mode) based on the remainingintra prediction mode information. The mpm list may be configured toinclude or not include a planar mode as a candidate. For example, if thempm list includes the planar mode as a candidate, the mpm list may havesix candidates. If the mpm list does not include the planar mode as acandidate, the mpm list may have three candidates. When the mpm listdoes not include the planar mode as a candidate, a not planar flag (ex.intra_luma_not_planar_flag) indicating whether an intra prediction modeof the current block is not the planar mode may be signaled. Forexample, the mpm flag may be signaled first, and the mpm index and notplanar flag may be signaled when the value of the mpm flag is 1. Inaddition, the mpm index may be signaled when the value of the not planarflag is 1. Here, the mpm list is configured not to include the planarmode as a candidate does not is to signal the not planar flag first tocheck whether it is the planar mode first because the planar mode isalways considered as mpm.

For example, whether the intra prediction mode applied to the currentblock is in mpm candidates (and planar mode) or in remaining mode may beindicated based on the mpm flag (ex. Intra_luma_mpm_flag). A value 1 ofthe mpm flag may indicate that the intra prediction mode for the currentblock is within mpm candidates (and planar mode), and a value 0 of thempm flag may indicate that the intra prediction mode for the currentblock is not in the mpm candidates (and planar mode). The value 0 of thenot planar flag (ex.

Intra_luma_not_planar_flag) may indicate that the intra prediction modefor the current block is planar mode, and the value 1 of the not planarflag value may indicate that the intra prediction mode for the currentblock is not the planar mode. The mpm index may be signaled in the formof an mpm_idx or intra_luma_mpm_idx syntax element, and the remainingintra prediction mode information may be signaled in the form of arem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. Forexample, the remaining intra prediction mode information may indexremaining intra prediction modes not included in the mpm candidates (andplanar mode) among all intra prediction modes in order of predictionmode number to indicate one of them. The intra prediction mode may be anintra prediction mode for a luma component (sample). Hereinafter, intraprediction mode information may include at least one of the mpm flag(ex. Intra_luma_mpm_flag), the not planar flag (ex.Intra_luma_not_planar_flag), the mpm index (ex. mpm_idx orintra_luma_mpm_idx), and the remaining intra prediction mode information(rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the presentdocument, the MPM list may be referred to in various terms such as MPMcandidate list and candModeList. When MIP is applied to the currentblock, a separate mpm flag (ex. intra_mip_mpm_flag), an mpm index (ex.intra_mip_mpm_idx), and remaining intra prediction mode information (ex.intra_mip_mpm_remainder) for MIP may be signaled and the not planar flagis not signaled.

In other words, in general, when block splitting is performed on animage, a current block and a neighboring block to be coded have similarimage characteristics. Therefore, the current block and the neighboringblock have a high probability of having the same or similar intraprediction mode. Thus, the encoder may use the intra prediction mode ofthe neighboring block to encode the intra prediction mode of the currentblock.

For example, the encoder/decoder may configure a list of most probablemodes (MPM) for the current block. The MPM list may also be referred toas an MPM candidate list. Herein, the MPM may refer to a mode used toimprove coding efficiency in consideration of similarity between thecurrent block and neighboring block in intra prediction mode coding. Asdescribed above, the MPM list may be configured to include the planarmode or may be configured to exclude the planar mode. For example, whenthe MPM list includes the planar mode, the number of candidates in theMPM list may be 6. And, if the MPM list does not include the planarmode, the number of candidates in the MPM list may be 5.

The encoder/decoder may configure an MPM list including 5 or 6 MPMs.

In order to configure the MPM list, three types of modes can beconsidered: default intra modes, neighbor intra modes, and the derivedintra modes.

For the neighboring intra modes, two neighboring blocks, i.e., a leftneighboring block and an upper neighboring block, may be considered.

As described above, if the MPM list is configured not to include theplanar mode, the planar mode is excluded from the list, and the numberof MPM list candidates may be set to 5.

In addition, the non-directional mode (or non-angular mode) among theintra prediction modes may include a DC mode based on the average ofneighboring reference samples of the current block or a planar modebased on interpolation.

When inter prediction is applied, the predictor of the encodingapparatus/decoding apparatus may derive a prediction sample byperforming inter prediction in units of blocks. Inter prediction may bea prediction derived in a manner that is dependent on data elements (ex.sample values or motion information) of picture(s) other than thecurrent picture. When inter prediction is applied to the current block,a predicted block (prediction sample array) for the current block may bederived based on a reference block (reference sample array) specified bya motion vector on the reference picture indicated by the referencepicture index. Here, in order to reduce the amount of motion informationtransmitted in the inter prediction mode, the motion information of thecurrent block may be predicted in units of blocks, subblocks, or samplesbased on correlation of motion information between the neighboring blockand the current block. The motion information may include a motionvector and a reference picture index. The motion information may furtherinclude inter prediction type (L0 prediction, L1 prediction, Biprediction, etc.) information. In the case of inter prediction, theneighboring block may include a spatial neighboring block present in thecurrent picture and a temporal neighboring block present in thereference picture. The reference picture including the reference blockand the reference picture including the temporal neighboring block maybe the same or different. The temporal neighboring block may be called acollocated reference block, a co-located CU (colCU), and the like, andthe reference picture including the temporal neighboring block may becalled a collocated picture (colPic). For example, a motion informationcandidate list may be configured based on neighboring blocks of thecurrent block, and flag or index information indicating which candidateis selected (used) may be signaled to derive a motion vector and/or areference picture index of the current block. Inter prediction may beperformed based on various prediction modes. For example, in the case ofa skip mode and a merge mode, the motion information of the currentblock may be the same as motion information of the neighboring block. Inthe skip mode, unlike the merge mode, the residual signal may not betransmitted. In the case of the motion vector prediction (MVP) mode, themotion vector of the selected neighboring block may be used as a motionvector predictor and the motion vector of the current block may besignaled. In this case, the motion vector of the current block may bederived using the sum of the motion vector predictor and the motionvector difference.

The motion information may include L0 motion information and/or L1motion information according to an inter prediction type (L0 prediction,L1 prediction, Bi prediction, etc.). The motion vector in the L0direction may be referred to as an L0 motion vector or MVL0, and themotion vector in the L1 direction may be referred to as an L1 motionvector or MVL1. Prediction based on the L0 motion vector may be calledL0 prediction, prediction based on the L1 motion vector may be called L1prediction, and prediction based on both the L0 motion vector and the L1motion vector may be called bi-prediction. Here, the L0 motion vectormay indicate a motion vector associated with the reference picture listL0 (L0), and the L1 motion vector may indicate a motion vectorassociated with the reference picture list L1 (L1). The referencepicture list L0 may include pictures that are earlier in output orderthan the current picture as reference pictures, and the referencepicture list L1 may include pictures that are later in the output orderthan the current picture. The previous pictures may be called forward(reference) pictures, and the subsequent pictures may be called reverse(reference) pictures. The reference picture list L0 may further includepictures that are later in the output order than the current picture asreference pictures. In this case, the previous pictures may be indexedfirst in the reference picture list L0 and the subsequent pictures maybe indexed later. The reference picture list L1 may further includeprevious pictures in the output order than the current picture asreference pictures. In this case, the subsequent pictures may be indexedfirst in the reference picture list 1 and the previous pictures may beindexed later. The output order may correspond to picture order count(POC) order.

A video/image encoding procedure based on inter prediction may include,for example, the following.

FIG. 4 shows an example of a video/image encoding method based on interprediction.

The encoding apparatus performs inter prediction on the current block(S400). The encoding apparatus may derive inter prediction mode andmotion information of the current block and generate prediction samplesof the current block. Here, the inter prediction mode determination, themotion information derivation, and the prediction samples generationprocedure may be performed simultaneously, or one procedure may beperformed before the other. For example, the inter predictor of theencoding apparatus may include a prediction mode determiner, a motioninformation deriver, and a prediction sample deriver. The predictionmode determiner may determine a prediction mode for the current block,the motion information deriver may derive motion information of thecurrent block, and prediction sample deriver may derive motion samplesof the current block. For example, the inter predictor of the encodingapparatus may search for a block similar to the current block in apredetermined region (search region) of reference pictures throughmotion estimation and derive a reference block whose difference to thecurrent block is a minimum or a predetermined reference or less. Basedon this, the inter predictor may derive a reference picture indexindicating a reference picture in which the reference block is locatedand derive a motion vector based on a position difference between thereference block and the current block. The encoding apparatus maydetermine a mode applied to the current block among various predictionmodes. The encoding apparatus may compare RD costs for the variousprediction modes and determine an optimal prediction mode for thecurrent block.

For example, when a skip mode or a merge mode is applied to the currentblock, the encoding apparatus may configure a merge candidate list to bedescribed later and derive a reference block whose difference to thecurrent block is minimum or a predetermined reference or less amongreference blocks indicated by the merge candidates included in the mergecandidate list. In this case, a merge candidate associated with thederived reference block may be selected, and merge index informationindicating the selected merge candidate may be generated and signaled tothe decoding apparatus. The motion information of the current block maybe derived using the motion information of the selected merge candidate.

As another example, when the (A)MVP mode is applied to the currentblock, the encoding apparatus may configure a (A)MVP candidate list tobe described later and use a motion vector of an mvp candidate selectedfrom among the mvp (motion vector predictor) candidates included in the(A)MVP candidate list, as mvp of the current block. In this case, forexample, a motion vector indicating the reference block derived by theabove-described motion estimation may be used as the motion vector ofthe current block, and an mvp candidate having a motion vector whosedifference to the motion vector of the current block, among the mvpcandidates, is smallest may be the selected mvp candidate. A motionvector difference (MVP) which is a difference from which the mvp wassubtracted may be derived from the motion vector of the current block.In this case, the information on the MVD may be signaled to the decodingapparatus. In addition, when the (A)MVP mode is applied, the value ofthe reference picture index may be configured as reference picture indexinformation and separately signaled to the decoding apparatus.

The encoding apparatus may derive residual samples based on theprediction samples (S410). The encoding apparatus may derive theresidual samples by comparing the original samples of the current blockwith the prediction samples.

The encoding apparatus encodes image information including predictioninformation and residual information (S420). The encoding apparatus mayoutput the encoded image information in the form of a bitstream. Theprediction information may include prediction mode information (ex. skipflag, merge flag or mode index) and information on motion information asinformation related to the prediction procedure. The information on themotion information may include candidate selection information (ex.merge index, mvp flag or mvp index) that is information for deriving amotion vector. In addition, the information on the motion informationmay include the information on the MVD and/or reference picture indexinformation described above. The information on the motion informationmay include information indicating whether L0 prediction, L1 prediction,or bi prediction is applied. The residual information is information onthe residual samples. The residual information may include informationon quantized transform coefficients for the residual samples.

The output bitstream may be stored in a (digital) storage medium anddelivered to the decoding apparatus, or may be delivered to the decodingapparatus via a network.

Meanwhile, as described above, the encoding apparatus may generate areconstructed picture (including the reconstructed samples and thereconstructed block) based on the reference samples and the residualsamples. This is to derive the same prediction result in the encodingapparatus as that performed in the decoding apparatus and because codingefficiency may be increased. Therefore, the encoding apparatus may storea reconstructed picture (or reconstructed samples, a reconstructedblock) in the memory and use it as a reference picture for interprediction. As described above, the in-loop filtering procedure may befurther applied to the reconstructed picture.

A video/image decoding procedure based on inter prediction may include,for example, the following.

FIG. 5 shows an example of a video/image decoding method based on interprediction.

Referring to FIG. 5, the decoding apparatus may perform an operationcorresponding to the operation performed in the encoding apparatus. Thedecoding apparatus may perform prediction on the current block based onthe received prediction information and derive prediction samples.

Specifically, the decoding apparatus may determine a prediction mode forthe current block based on the received prediction information (S500).The decoding apparatus may determine which inter prediction mode isapplied to the current block based on the prediction mode information inthe prediction information.

For example, the decoding apparatus may determine whether the merge modeis applied to the current block or whether (A)MVP mode is determinedbased on the merge flag. Alternatively, one of various inter predictionmode candidates may be selected based on the mode index. The interprediction mode candidates may include a skip mode, a merge mode, and/or(A)MVP mode, or may include various inter prediction modes describedbelow.

The decoding apparatus derives motion information of the current blockbased on the determined inter prediction mode (S510). For example, whena skip mode or a merge mode is applied to the current block, thedecoding apparatus may configure a merge candidate list to be describedlater, and select one of the merge candidates included in the mergecandidate list. The selection may be performed based on the aboveselection information (merge index). The motion information of thecurrent block may be derived using the motion information of theselected merge candidate. The motion information of the selected mergecandidate may be used as motion information of the current block.

As another example, when the (A)MVP mode is applied to the currentblock, the decoding apparatus may configure an (A)MVP candidate list tobe described later and use a motion vector of an mvp candidate selectedfrom the mvp candidates included in the (A)MVP candidate list as mvp ofthe current block. The selection may be performed based on theabove-described selection information (mvp flag or mvp index). In thiscase, the MVD of the current block may be derived based on theinformation on the MVD, and the motion vector of the current block maybe derived based on mvp and the MVD of the current block. In addition, areference picture index of the current block may be derived based on thereference picture index information. A picture indicated by thereference picture index in the reference picture list for the currentblock may be derived as a reference picture referred for interprediction of the current block.

Meanwhile, as described below, motion information of the current blockmay be derived without configuring a candidate list, and in this case,motion information of the current block may be derived according to aprocedure disclosed in a prediction mode to be described later. In thiscase, the configuration of the candidate list as described above may beomitted.

The decoding apparatus may generate prediction samples for the currentblock based on the motion information of the current block (S520). Inthis case, the reference picture may be derived based on the referencepicture index of the current block, and the prediction samples of thecurrent block may be derived using the samples of the reference blockindicated by the motion vector of the current block on the referencepicture. In this case, as described below, a prediction sample filteringprocedure may be further performed on all or some of the predictionsamples of the current block.

For example, the inter predictor of the decoding apparatus may include aprediction mode determiner, a motion information deriver, and aprediction sample deriver. The prediction mode for the current block maybe determined based on the prediction mode information received from theprediction mode determiner, motion information (motion vector and/orreference picture index, etc.) of the current block may be derived basedon the information on the motion information received from the motioninformation deriver, and the prediction sample deriver may derive theprediction samples of the current block.

The decoding apparatus generates residual samples for the current blockbased on the received residual information (S530). The decodingapparatus may generate reconstructed samples for the current block basedon the prediction samples and the residual samples and generate areconstructed picture based on the reconstructed samples (S540).Thereafter, the in-loop filtering procedure or the like may be furtherapplied as described above.

FIG. 6 exemplarily shows an inter prediction procedure.

Referring to FIG. 6, as described above, the inter prediction proceduremay include determining an inter prediction mode, deriving motioninformation according to the determined prediction mode, and performingprediction based on the derived motion information (generation of aprediction sample). The inter prediction procedure may be performed bythe encoding apparatus and the decoding apparatus as described above. Inthis document, a coding apparatus may include an encoding apparatusand/or a decoding apparatus.

Referring to FIG. 6, the coding apparatus determines an inter predictionmode for the current block (S600). Various inter prediction modes may beused for prediction of the current block in the picture. For example,various modes, such as a merge mode, a skip mode, a motion vectorprediction (MVP) mode, an affine mode, a subblock merge mode, and amerge with MVD (MMVD) mode, and the like may be used. A decoder sidemotion vector refinement (DMVR) mode, an adaptive motion vectorresolution (AMVR) mode, a bi-prediction with CU-level weight (BCW), abi-directional optical flow (BDOF), and the like may also be used asadditional modes additionally or instead. The affine mode may be calledan affine motion prediction mode. The MVP mode may be referred to asadvanced motion vector prediction (AMVP) mode. In this document, somemodes and/or motion information candidates derived by some modes may beincluded as one of motion information candidates of other modes. Forexample, an HMVP candidate may be added as a merge candidate in themerge/skip mode or may be added as an MVP candidate in the MVP mode.When the HMVP candidate is used as a motion information candidate in themerge mode or the skip mode, the HMVP candidate may be referred to as anHMVP merge candidate.

Prediction mode information indicating the inter prediction mode of thecurrent block may be signaled from the encoding apparatus to thedecoding apparatus. The prediction mode information may be included inthe bitstream and received by the decoding apparatus. The predictionmode information may include index information indicating one of aplurality of candidate modes. Alternatively, the inter prediction modemay be indicated through hierarchical signaling of flag information. Inthis case, the prediction mode information may include one or moreflags. For example, a skip flag may be signaled to indicate whether askip mode is applied, and if the skip mode is not applied, a merge flagmay be signaled to indicate whether a merge mode is applied, and if themerge mode is not applied, it is indicated to apply an MVP mode or aflag for additional classification may be further signaled. The affinemode may be signaled in an independent mode or may be signaled in a modedependent on the merge mode or the MVP mode. For example, the affinemode may include an affine merge mode and an affine MVP mode.

Meanwhile, information indicating whether the list0 (L0) prediction, thelist1 (L1) prediction, or the bi-prediction described above is used inthe current block (current coding unit) may be signaled in the currentblock. The information may be referred to as motion prediction directioninformation, inter prediction direction information or inter predictionindication information, and may be configured/encoded/signaled in theform of, for example, an inter_pred_idc syntax element. That is, theinter_pred_idc syntax element may indicate whether the aforementionedlist0 (L0) prediction, list1 (L1) prediction, or bi-prediction is usedfor the current block (current coding unit). In this document, for theconvenience of description, the inter prediction type (L0 prediction, L1prediction, or BI prediction) indicated by the inter_pred_idc syntaxelement may be indicated as a motion prediction direction. L0 predictionmay be represented as pred_L0, L1 prediction as pred_L1, andbi-prediction as pred_BI. For example, the following prediction type maybe indicated according to the value of the inter_pred_idc syntaxelement.

TABLE 1 Name of inter_pred_idc Value of ( cbWidth + ( cbWidth +inter_pred_idc cbHeight ) != 8 cbHeight ) = = 8 0 PRED_L0 PRED_L0 1PRED_L1 PRED_L1 2 PRED_BI —

As described above, one picture may include one or more slices. Theslice may have one of slice types including intra (I) slice, predictive(P) slice, and bi-predictive (B) slice. The slice type may be indicatedbased on slice type information. For blocks in an I slice, interprediction may not be used for prediction and only intra prediction maybe used. Of course, even in this case, the original sample value may becoded and signaled without prediction. Intra prediction or interprediction may be used for blocks in a P slice, and only uni predictionmay be used when inter prediction is used. Meanwhile, intra predictionor inter prediction may be used for blocks in a B slice, and up to biprediction may be used when inter prediction is used.

L0 and L1 may include reference pictures that are previouslyencoded/decoded prior to the current picture. For example, L0 mayinclude reference pictures before and/or after the current picture inPOC order, and L1 may include reference pictures after and/or before thecurrent picture in POC order. In this case, L0 may be assigned a lowerreference picture index relative to previous reference pictures in thePOC order than the current reference pictures, and L1 may be assigned alower reference picture index relative to previous reference pictures inthe POC order than the current picture. In the case of B slice,bi-prediction may be applied, and in this case, unidirectionalbi-prediction may be applied or bidirectional bi-prediction may beapplied. The bidirectional bi-prediction may be called truebi-prediction.

The following table shows syntax for a coding unit according to anembodiment of this document.

TABLE 2 coding_unit( x0, y0, cbWidth, cbHeight, treeType ) { Descriptor if( slice_type != I | | sps_ibc_enabled_flag ) {   if( treeType !=DUAL_TREE_CHROMA &&    !( cbWidth = = 4 && cbHeight = = 4 &&!sps_ibc_enabled_flag ) )     cu_skip_flag[ x0 ][ y0 ] ae(v)   if(cu_skip_flag[ x0 ][ y0 ] = = 0 && slice_type != I    && !( cbWidth = = 4&& cbHeight = = 4 ) )     pred_mode_flag ae(v)   if( ( ( slice_type = =I && cu_skip_flag[ x0 ][ y0 ] = =0) | |    ( slice_type != I && (CuPredMode[ x0 ][ y0 ] != MODE_1NTRA | |    ( cbWidth = = 4 && cbHeight= = 4 && cu_skip_flag[ x0 ][ y0 ] = = 0) ) ) ) &&   sps_ibc_enabled_flag && ( cbWidth != 128 | | cbHeight !=128 ) )   pred_mode_ibc_flag ae(v)  }  if( CuPredMode[ x0 ][ y0 ] = =MODE_INTRA ) {   if( sps_pcm_enabled_flag &&    cbWidth >=MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&    cbHeight >=MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY )    pcm_flag[ x0 ][ y0 ]ae(v)   if( pcm_flag[ x0 ][ y0 I] ) {    while( !byte_aligned( ) )    pcm_alignment_zero_bit f(1)    pcm_sample( cbWidth, cbHeight,treeType)   } else {    if( treeType = = SINGLE_TREE | | treeType = =DUAL_TREE_LUMA ) {     if( cbWidth <= 32 && cbHeight <= 32 )     intra_bdpcm_flag[ x0 ][ y0 ] ae(v)     if( intra bdpcm_flag[ x0 ][y0 ] )      intra_bdpcm_dir_flag[ x0 ][ y0 ] ae(v)     else {      if(sps_mip_enabled_flag &&       ( Abs( Log2( cbWidth ) − Log2( cbHeight )) <= 2 ) &&        cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY )     intra_mip_flag[ x0 ][ y0 ] ae(v)     if( intra_mip_flag[ x0 ][ y0 ]) {       intra_mip_mpm_flag[ x0 ][ y0 ] ae(v)      if(intra_mip_mpm_flag[ x0 ][ y0 ] )       intra_mip_mpm_idx[ x0 ][ y0 ]ae(v)      else       intra_mip_mpm_remainder[ x0 ][ y0 ] ae(v)     }else {      if( sps_mrl_enabled flag && ( ( y0 % CtbSizeY ) > 0 ) )      intra_luma_ref_idx[ x0 ][ y0 ] ae(v)       if (sps_isp_enabled_flag && intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&        (cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) &&        ( cbWidth *cbHeight > MinTbSizeY * MinTbSizeY ) )       intra_subpartitions_mode_flag[ x0 ][ y0] ae(v)       if(intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&        cbWidth <=MaxTbSizeY && cbHeight <= MaxTbSizeY )       intra_subpartition_split_flag[ x0 ][ y0 ] ae(v)       if(intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&       intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 )       intra_luma_mpm_flag[ x0 ][ y0 ] ae(v)       if(intra_luma_mpm_flag[ x0 ][ y0] ) {        if( intra_luma_ref_idx[ x0 ][y0 ] = = 0 )         intra_luma_not_planar_flag[ x0 ][ y0 ] ae(v)       if( intra_luma_not_planar_flag[ x0 ][y0 ] )        intra_luma_mpm_idx[ x0 ][ y0 ] ae(v)       } else       intra_luma_mpm_remainder[ x0 ][ y0 ] ae(v)      }     }    }   if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA )    intra_chroma_pred_mode[ x0 ][ y0 ] ae(v)   }  } else if( treeType !=DUAL_TREE_CHROMA ) { /* MODE_INTER or MODE_IBC */   if( cu_skip_flag[ x0][ v0 ] = = 0 )    general_merge_flag[ x0 ][ y0 ] ae(v)   if(general_merge_flag[ x0 ][ y0 ] ) {    merge_data( x0, y0, cbWidth,cbHeight )   } else if ( CuPredMode[ x0 ][ y0 ] = = MODE_IBC ) {   mvd_coding( x0, y0, 0, 0 )    mvp_l0_flag[ x0 ][ y0 ] ae(v)    if(sps_amvr_enabled_flag &&     ( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0][ y0 ][ 1 ] != 0 ) ) {     amvr_precision_flag[ x0 ][ y0 ] ae(v)    }  } else {    if( slice_type = = B )     inter_pred_idc[ x0 ][ y0 ]ae(v)    if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16) {     inter_affine_flag[ x0 ][ y0 ] ae(v)     if( sps_affine_type_flag&& inter_affine_flag[ x0 ][ y0 ] )      cu_affine_type_flag[ x0 ][ y0 ]ae(v)    }    if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] == PRED_BI &&     !inter_affine_flag[ x0 ][ y0 ] && RefIdxSymL0 > − 1 &&RefIdxSymL1 > − 1 )     sym_mvd_flag[ x0 ][ y0 ] ae(v)    if(inter_pred_idc[ x0 ][ y0 ] != PRED_L1 ) {     if( NumRefldxActive[ 0 ] >1 && !sym_mvd_flag[ x0 ][ y0 ] )      ref_idx_l0[ x0 ][ y0 ] ae(v)    mvd_coding( x0, y0, 0, 0 )     if( MotionModelIdc[ x0 ][ y0 ] > 0 )     mvd_coding( x0, y0, 0, 1 )     if(MotionModelIdc[ x0 ][ y0 ] > 1 )     mvd_coding( x0, y0, 0, 2 )     mvp_l0_flag[ x0 ][ y0 ] ae(v)    }else {     MvdL0[ x0 ][ y0 ][ 0 ] = 0     MvdL0[ x0 ][ y0 ][ 1 ] = 0   }    if( inter_pred_idc[ x0 ][ y0 ] != PRED_L0 ) {     if(NumRefIdxActive[ 1 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ] )      ref_idx_l1[x0 ][ y0 ] ae(v)     if( mvd_l1_zero_flag && inter_pred_idc[ x0 ][ y0 ]= = PRED_BI ) {      MvdL1[ x0 ][ y0 ][ 0 ] = 0      MvdL1[ x0 ][ y0 ][1 ] = 0      MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ] = 0      MvdCpL1[ x0 ][ y0 ][0 ][ 1 ] = 0      MvdCpL1[ x0 ][ y0 ][ 1 ][ 0 ] = 0      MvdCpL1[ x0 ][y0 ][ 1 ][ 1 ] = 0      MvdCpL1[ x0 ][ y0 ][ 2 ][ 0 ] = 0      MvdCpL1[x0 ][ y0 ][ 2 ][ 1 ] = 0     } else {      if( sym_mvd_flag[ x0 ][ y0 ]) {       MvdL1[ x0 ][ y0 ][ 0 ] = −MvdL0[ x0 ][ y0 ][ 0 ]       MvdL1[x0 ][ y0 ][ 1 ] = −MvdL0[ x0 ][ y0 ][ 1 ]      } else       mvd_coding(x0, y0, 1, 0 )      if( MotionModelIdc[ x0 ][ y0 ] > 0 )      mvd_coding( x0, y0, 1, 1 )      if(MotionModelIdc[ x0 ][ y0 ] > 1)       mvd_coding( x0, y0, 1, 2 )      mvp_l1_flag[ x0 ][ y0 ] ae(v)    }    } else {     MvdL1[ x0 ][ y0 ][ 0 ] = 0     MvdL1[ x0 ][ y0 ][1 ] = 0    }    if( ( sps_amvr_enabled_flag && inter_affine_flag[ x0 ][y0 ] = = 0 &&     ( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0 ][ y0 ][ 1] ! = 0 | |      MvdL1[ x0 ][ y0 ][ 0 ] != 0 | | MvdL1[ x0 ][ y0 ][ 1 ]!= 0 ) ) | |     ( sps_affine_amvr_enabled_flag && inter_affine_flag[ x0][ y0 ] = = 1 &&      ( MvdCpL0[ x0 ][ y0 ][ 0 ][ 0 ] != 0 | | MvdCpL0[x0 ][ y0 ][ 0 ][ 1 ] != 0 | |       MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ] != 0 || MvdCpL1[ x0 ][ y0 ][ 0 ][ 1 ] != 0 | |       MvdCpL0[ x0 ][ y0 ][ 1 ][0 ] != 0 | | MvdCpL0[ x0 ][ y0 ][ 1 ][ 1 ] != 0 | |       MvdCpL1[ x0 ][y0 ][ 1 ][ 0 ] != 0 | | MvdCpLl[ x0 ][ y0 ][ 1 ][ 1 ] != 0 | |      MvdCpL0[ x0 ][ y0 ][ 2 ][ 0 ] != 0 | | MvdCpL0[ x0 ][ y0 ][ 2 ][ 1] != 0 | |       MvdCpL1[ x0 ][ y0 ][ 2 ][ 0 ] != 0 | | MvdCpL1[ x0 ][y0 ][ 2 ] [ 1 ] != 0 ) ) {     amvr_flag[ x0 ][ y0 ] ae(v)     if(amvr_flag[ x0 ][ y0 ] )      amvr_precision_flag[ x0 ][ y0 ] ae(v)    }    if( sps_bcw_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = = PRED_BI&&      luma_weight_l0_flag[ ref_idx_l0 [ x0 ][ y0 ] ] = = 0 &&     luma_weight_l1_flag[ ref_idx_l1 [ x0 ][ y0 ] ] = = 0 &&     chroma_weight_l0_flag[ ref_idx_l0 [ x0 ][ y0 ] ] = = 0 &&     chroma_weight_l1_flag[ ref_idx_l1 [ x0 ][ y0 ] ] = = 0 &&     cbWidth * cbHeight >= 256 )     bcw_idx[ x0 ][ y0 ] ae(v)    }   }  if( !pcm_flag[ x0 ][ y0 ] ) {    if( CuPredMode[ x0 ][ y0 ] !=MODE_INTRA &&     general_merge_flag[ x0 ][ y0 ] = = 0 )     cu_cbfae(v)   if( cu_cbf ) {    if( CuPredMode[ x0 ][ y0 ] = = MODE_INTER &&sps_sbt_enabled_flag &&     !ciip_flag[ x0 ][ y0 ] &&!MergeTriangleFlag[ x0 ][ y0 ] ) {     if( cbWidth <= MaxSbtSize &&cbHeight <= MaxSbtSize ) {      allowSbtVerH = cbWidth >= 8     allowSbtVerQ = cbWidth >= 16      allowSbtHorH = cbHeight >= 8     alIowSbtHorQ = cbHeight >= 16      if( allowSbtVcrH | |allowSbtHorH | | allowSbtVcrQ | | allowSbtHorQ )       cu_sbt_flag ae(v)    }     if( cu_sbt_flag) {      if( ( allowSbtVcrH | | allowSbtHorH )&& ( allowSbtVcrQ | | allowSbtHorQ) )       cu_sbt_quad_flag ae(v)     if( ( cu_sbt_quad_flag && allow SbtVerQ && allowSbtHorQ ) | |      ( !cu_sbt_quad_flag && allowSbtVerH && allowSbtHorH ) )      cu_sbt_horizontal_flag ae(v)      cu_sbt_pos_flag ae(v)     }    }   numSigCoeff = 0    numZeroOutSigCoeff = 0    transform_tree( x0, y0,cbWidth, cbHeight, treeType )    lfnstWidth = ( treeType = =DUAL_TREE_CHROMA ) ? cbWidth / SubWidthC           : cbWidth   lfnstHeight = ( treeType = = DUAL_TREE_CHROMA ) ? cbHeight /SubHeightC           : cbHeight    if( Min( lfnstWidth, lfnstHeight ) >=4 && sps_lfnst_enabled_flag = = 1 &&     CuPredMode[ x0 ][ y0 ] = =MODE_INTRA &&     IntraSubPartitionsSplitType = = ISP_NO_SPLIT &&    !intra_mip_flag[ x0 ][ y0 ] ) {     if( ( numSigCoeff > ( ( treeType= = SINGLE_TREE) ? 2 : 1 ) ) &&       numZeroOutSigCoeff = = 0 )     lfnst_idx[ x0 ][ y0 ] ae(v)    }   }  } }

Referring to Table 2, general-merge-fag may indicate that regular mergeis available, and when the value of general_merge_flag is 1, a regularmerge mode, an mmvd mode, and a merge subblock mode (subblock mergemode) may be available. For example, when the value ofgeneral_merge_flag is 1, merge data syntax may be parsed from encodedvideo/image information (or bitstream), and the merge data syntax may beconfigured/coded to include the information as shown in the followingtable.

TABLE 3 Descriptor merge_data( x0, y0, cbWidth, cbHeight ) {  if(CuPredMode[ x0 ][ y0 ] == MODE_IBC ) {   if( MaxNumMergeCand > 1 )   merge_idx[ x0 ][ y0 ] ae(v)  } else {   if( sps_mmvd_enabled_flag | |cbWidth * cbHeight != 32 )    regular_merge_flag[ x0 ][ y0 ] ae(v)   if( regular_merge_flag[ x0 ][ y0 ] == 1 ) {    if( MaxNumMergeCand > 1 )    merge_idx[ x0 ][ y0 ] ae(v)   } else {    if( sps_mmvd_enabled_flag&& cbWidth * cbHeight != 32 )     mmvd_merge_flag[ x0 ][ y0 ] ae(v)   if( mmvd_merge_flag[ x0 ][ y0 ] == 1 ) {     if( MaxNumMergeCand > 1)      mmnvd_cand_flag[ x0 ][ y0 ] ae(v)     mmvd_distance_idx[ x0 ][ y0] ae(v)     mmvd_direction_idx[ x0 ][ y0 ] ae(v)    } else {     if(MaxNumSubblockMergeCand > 0 && cbWidth >=8 && cbHeight >= 8 )     merge_subblock_flag[ x0 ][ y0 ] ae(v)     if( merge_subblock_flag[x0 ][ y0 ] == 1 ) {      if( MaxNumSubblockMergeCand > 1 )      merge_subblock idx[ x0 ][ y0 ] ae(v)     } else {      if(sps_ciip_enabled_flag && cu_skip_flag[ x0 ][ y0 ] == 0 &&       (cbWidth * cbHeight ) >= 64 && cbWidth < 128 && cbHeight < 128 ) {      ciip_flag[ x0 ][ y0 ] ae(v)      if( ciip_flag[ x0 ][ y0 ] &&MaxNumMergeCand > 1 )       merge_idx[ x0 ][ y0 ] ae(v)      }      if(MergeTriangleFlag[ x0 ][ y0 ] ) {       merge_triangle_split_dir[ x0 ][y0 ] ae(v)       merge_triangle_idx0[ x0 ][ y0 ] ae(v)      merge_triangle_idx1[ x0 ][ y0 ] ae(v)      }     }    }   }  } }

The coding apparatus derives motion information for the current block(S610). The motion information derivation may be derived based on theinter prediction mode.

Inter prediction may be performed using motion information of thecurrent block. The encoding apparatus may derive optimal motioninformation for the current block through a motion estimation procedure.For example, the encoding apparatus may search for a similar referenceblock having a high correlation in units of fractional pixels within apredetermined search range in the reference picture using the originalblock in the original picture for the current block, thereby derivingmotion information. Similarity of blocks may be derived based on adifference of phase based sample values. For example, the similarity ofthe blocks may be calculated based on the SAD between the current block(or template of the current block) and the reference block (or templateof the reference block). In this case, motion information may be derivedbased on a reference block having the smallest SAD in the search area.The derived motion information may be signaled to the decoding apparatusaccording to various methods based on the inter prediction mode.

The coding apparatus performs inter prediction based on the motioninformation on the current block (S620). The coding apparatus may deriveprediction sample(s) for the current block based on the motioninformation. The current block including the prediction samples may bereferred to as a predicted block.

When the merge mode is applied, the motion information of the currentprediction block is not directly transmitted, and the motion informationof the current prediction block is derived using motion information of aneighboring prediction block. Therefore, the motion information of thecurrent prediction block may be indicated by transmitting flaginformation indicating that the merge mode is used and a merge indexindicating which neighboring prediction blocks are used. The merge modemay be called a regular merge mode.

The encoder must search a merge candidate block used to derive motioninformation of the current prediction block to perform the merge mode.For example, up to five merge candidate blocks may be used, but theembodiment(s) of the present document are not limited thereto. A maximumnumber of the merge candidate blocks may be transmitted in a sliceheader or a tile group header and the embodiment(s) of the presentdocument are not limited thereto. After finding the merge candidateblocks, the encoder may generate a merge candidate list and select amerge candidate block having the smallest cost among them as a finalmerge candidate block.

The merge candidate list may use, for example, five merge candidateblocks. For example, four spatial merge candidates and one temporalmerge candidate may be used. Hereinafter, the spatial merge candidate orthe spatial MVP candidate to be described later may be referred to asSMVP, and the temporal merge candidate or the temporal MVP candidate tobe described later may be referred to as TMVP.

Hereinafter, a method of constructing a merge candidate list accordingto this document is described.

The coding apparatus (encoder/decoder) inserts spatial merge candidatesderived by searching for spatial neighboring blocks of the current blockinto the merge candidate list. For example, the spatial neighboringblocks may include a bottom left corner neighboring block, a leftneighboring block, a upper right corner neighboring block, an upperneighboring block, and an upper left corner neighboring block of thecurrent block. However, this is an example, and in addition to theabove-described spatial neighboring blocks, additional neighboringblocks such as a right neighboring block, a bottom neighboring block,and a bottom right neighboring block may be further used as the spatialneighboring blocks. The coding apparatus may detect available blocks bysearching the spatial neighboring blocks based on the priority, and mayderive motion information of the detected blocks as the spatial mergecandidates.

The coding apparatus inserts the temporal merge candidate derived bysearching the temporal neighboring block of the current block into themerge candidate list. The temporal neighboring block may be located on areference picture that is a picture different from the current picturein which the current block is located. The reference picture in whichthe temporal neighboring block is located may be called a collocatedpicture or a col picture. The temporal neighboring block may be searchedin order of the bottom right corner neighboring block and the bottomright center block of the co-located block for the current block on thecol picture. Meanwhile, when motion data compression is applied,specific motion information may be stored as representative motioninformation for each predetermined storage unit in the col picture. Inthis case, it is not necessary to store the motion information for allthe blocks in the predetermined storage unit, thereby obtaining a motiondata compression effect. In this case, the predetermined storage unitmay be previously determined, for example, in 16×16 sample units, 8×8sample units, or the like, or size information on the predeterminedstorage unit may be signaled from the encoder to the decoder. When themotion data compression is applied, motion information of the temporalneighboring block may be replaced with representative motion informationof the predetermined storage unit in which the temporal neighboringblock is located. That is, in this case, from an implementation point ofview, a predetermined value is arithmetically shifted to the right basedon coordinates (top left sample position) of the temporal neighboringblock, and thereafter, the temporal merge candidate may be derived basedon motion information of the prediction block covering an arithmeticallyleft shifted position. For example, in the case of a sample unit havingthe predetermined storage unit is 2n×2n, if the coordinates of thetemporal neighboring block are (xTnb, yTnb), motion information of theprediction block located at the modified position ((xTnb>>n)<<n),(yTnb>>n)<<n)). Specifically, for example, in case where thepredetermined storage unit is a 16×16 sample unit, if the coordinates ofthe temporal neighboring block are (xTnb, yTnb), motion information ofthe prediction block located at modified position ((xTnb>>4)<<4),(yTnb>>4)<<4)) may be used for the temporal merge candidate. Or, forexample, in case where the predetermined storage unit is an 8×8 sampleunit, if the coordinates of the temporal neighboring block are (xTnb,yTnb), motion information of the prediction block located at themodified position ((xTnb>>3)<<3), (yTnb>>3)<<3)) may be used for thetemporal merge candidate.

The coding apparatus may determine whether the number of current mergecandidates is smaller than the maximum number of merge candidates. Themaximum number of merge candidates may be predefined or signaled fromthe encoder to the decoder. For example, the encoder may generateinformation on the maximum number of merge candidates, encode theinformation, and transmit the encoded information to the decoder in theform of a bitstream. If the maximum number of merge candidates is filledup, a subsequent candidate addition process may not be performed.

As a result of the checking, if the number of the current mergecandidates is smaller than the maximum number of merge candidates, thecoding apparatus inserts the additional merge candidate into the mergecandidate list.

As a result of the checking, if the number of the current mergecandidates is not smaller than the number of the maximum mergecandidates, the coding apparatus may terminate the construction of themerge candidate list. In this case, the encoder may select an optimalmerge candidate among merge candidates configuring the merge candidatelist based on a rate-distortion (RD) cost, and signal selectioninformation (ex. merge index) indicating the selected merge candidate tothe decoder. The decoder may select the optimal merge candidate based onthe merge candidate list and the selection information.

The motion information of the selected merge candidate may be used asthe motion information of the current block, and the prediction samplesof the current block may be derived based on the motion information ofthe current block. An encoder may derive residual samples of the currentblock based on the prediction samples, and may signal residualinformation on the residual samples to a decoder. The decoder maygenerate reconstructed samples based on the residual samples and thepredicted samples derived based on the residual information, andgenerate a reconstructed picture based thereon as described above.

When the skip mode is applied, the motion information of the currentblock may be derived in the same manner as that of the case where themerge mode is applied. However, when the skip mode is applied, theresidual signal for the corresponding block is omitted, and thusprediction samples may be used as reconstructed samples.

When the MVP mode is applied, a motion vector predictor (mvp) candidatelist may be generated using a motion vector of a reconstructed spatialneighboring block and/or a motion vector of a temporal neighboring block(or Col block). That is, the motion vector corresponding to thereconstructed spatial neighboring block and/or the motion vectorcorresponding to the temporal neighboring block may be used as a motionvector predictor candidate. When bi-prediction is applied, an mvpcandidate list for deriving L0 motion information and an mvp candidatelist for deriving L1 motion information may be generated and usedseparately. The above-described prediction information (or informationon the prediction) may include selection information (ex. MVP flag orMVP index) indicating an optimal motion vector predictor candidateselected from the motion vector predictor candidates included in thelist. In this case, the predictor may select a motion vector predictorof the current block from among the motion vector predictor candidatesincluded in the motion vector candidate list using the selectioninformation. The predictor of the encoding apparatus may obtain a motionvector difference (MVD) between the motion vector of the current blockand the motion vector predictor, encode the same, and output it in abitstream form. That is, the MVD may be obtained as a value obtained bysubtracting the motion vector predictor from the motion vector of thecurrent block. In this case, the predictor of the decoding apparatus mayobtain a motion vector difference included in the information on theprediction and derive the motion vector of the current block by addingthe motion vector difference and the motion vector predictor. Thepredictor of the decoding apparatus may obtain or derive a referencepicture index indicating the reference picture from the information onthe prediction.

Hereinafter, a method of constructing a motion vector predictorcandidate list according to this document is described.

One embodiment may first search for a spatial candidate block for motionvector prediction and insert it into the prediction candidate list.Thereafter, an embodiment may determine whether the number of spatialcandidate blocks is less than two. For example, in an embodiment, whenthe number of spatial candidate blocks is less than 2, a temporalcandidate block may be searched for and additionally inserted into theprediction candidate list, and when the temporal candidate block isunavailable, a zero motion vector may be used. That is, the zero motionvector may be additionally inserted into the prediction candidate list.Thereafter, an embodiment may end the construction of the preliminarycandidate list. Alternatively, according to an embodiment, when thenumber of spatial candidate blocks is not less than two, theconstruction of the preliminary candidate list may be terminated. Here,the preliminary candidate list may indicate an MVP candidate list.

Meanwhile, when the MVP mode is applied, the reference picture index maybe explicitly signaled. In this case, the reference picture indexrefidxL0 for the L0 prediction and the reference picture index refidxL1for the L1 prediction may be separately signaled. For example, when MVPmode is applied and BI prediction is applied, both information onrefidxL0 and information on refidxL1 may be signaled.

When the MVP mode is applied, as described above, the information on theMVD derived from the encoding apparatus may be signaled to the decodingapparatus. The information on the MVD may include, for example,information representing x and y components of the MVD absolute valueand the sign. In this case, information indicating whether the MVDabsolute value is greater than 0 and greater than 1, and the MVDremainder may be signaled step by step. For example, the informationindicating whether the MVD absolute value is greater than 1 may besignaled only when the value of the flag information indicating whetherthe MVD absolute value is greater than 0 is 1.

For example, the information on the MVD may be configured as thefollowing syntax, encoded in the encoding apparatus, and signaled to thedecoding apparatus.

TABLE 4 Descriptor mvd_coding( x0, y0 refList ,cpIdx ) { abs_mvd_greater0_flag[ 0 ] ae(v)  abs_mvd_greater0_flag[ 1 ] ae(v)  if(abs_mvd_greater0_flag[ 0 ] )   abs_mvd_greater1_flag[ 0 ] ae(v)  if(abs_mvd_greater0_flag[ 1 ] )   abs_mvd_greater1_flag[ 1 ] ae(v)  if(abs_mvd_greater0_flag[ 0 ] ) {   if( abs_mvd_greater1_flag[ 0 ] )   abs_mvd_minus2[ 0 ] ae(v)   mvd_sign_flag[ 0 ] ae(v)  }  if(abs_mvd_greater0_flag[ 1 ] ) {   if( abs_mvd_greater1_flag[ 1 ] )   abs_mvd_minus2[ 1 ] ae(v)   mvd_sign_flag[ 1 ] ae(v)  } }

For example, in Table 4, the abs_mvd_greater0_flag syntax element mayindicate information on whether the difference MVD is greater than 0,and the abs_mvd_greater1_flag syntax element may indicate information onwhether the difference MVD is greater than 1. Also, the abs_mvd_minus2syntax element may indicate information about a value obtained by −2 tothe difference MVD, and the mvd_sign_flag syntax element may indicateinformation about the sign of the difference MVD. In addition, in Table4, [0] of each syntax element may indicate information on L0, and [1]may indicate information on L1.

For example, MVD[compIdx] may be derived based onabs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)*(1−2*mvd_sign_flag[compIdx]). Here, compIdx(or cpIdx) represents an index of each component and may have a value of0 or 1. compIdx 0 may indicate x component and compIdx 1 may indicate ycomponent. However, this is merely an example and values of eachcomponent may be expressed by using a coordinate system other than the xand y coordinate systems.

Meanwhile, MVD (MVDL0) for L0 prediction and MVD (MVDL1) for L1prediction may be separately signaled, and the information on MVD mayinclude information on MVDL0 and/or information on MVDL1. For example,when the MVP mode is applied to the current block and BI prediction isapplied, both information on the MVDL0 and information on the MVDL1 maybe signaled.

FIG. 7 is a diagram for describing symmetric motion vector differences(SMVD).

When BI prediction is applied, symmetric MVD may be used inconsideration of coding efficiency. In this case, signaling of some ofthe motion information may be omitted. For example, when symmetric MVDis applied to the current block, information on refidxL0, information onrefidxL1, and information on MVDL1 may not be signaled from the encodingapparatus to the decoding apparatus and may be internally derived. Forexample, when MVP mode and BI prediction are applied to the currentblock, flag information (ex. symmetric MVD flag information orsym_mvd_flag syntax element) indicating whether to apply symmetric MVDmay be signaled, and when the value of the flag information is 1, thedecoding apparatus may determine that symmetric MVD is applied to thecurrent block.

When symmetric MVD mode is applied (i.e., the value of symmetric MVDflag information is 1), information on mvp_10_flag, mvp_11_flag, andMVDL0 may be explicitly signaled and, as described above, signaling ofinformation on refidxL0, information on refidxL1, and information onMVDL1 may be omitted and derived internally. For example, refidxL0 maybe derived as an index indicating a previous reference picture closestto the current picture in POC order in reference picture list 0 (whichmay be called list 0 or L0). refidxL1 may be derived as an indexindicating a next reference picture closest to the current picture inthe POC order in reference picture list 1 (which may be called list 1 orL1). Or, for example, refidxL0 and refidxL1 may both be derived as 0.Or, for example, the refidxL0 and refidxL1 may be derived as minimumindexes having the same POC difference in the relationship with thecurrent picture. Specifically, for example, when [POC of currentpicture]-[POC of first reference picture indicated by refidxL0] is afirst POC difference and [POC of the second reference picture indicatedby refidxL1] is a second POC difference, only if the first POCdifference and the second POC difference are the same, a value ofrefidxL0 indicating the first reference picture may be derived as avalue of refidxL0 of the current block and a value of refidxL1indicating the second reference picture may be derived as a value ofrefidxL1 of the current block. In addition, for example, when there area plurality of sets in which the first POC difference and the second POCdifference are the same, refidxL0 and refidxL1 of a set having theminimum difference may be derived as refidxL0 and refidxL1 of thecurrent block.

Referring to FIG. 7, reference picture list 0, reference picture list 1,and MVDL0 and MVDL1 are shown. Here, MVDL1 is symmetric with MVDL0.

MVDL1 may be derived as negative (−) MVDL0. For example, the final(improved or modified) motion information (motion vector; MV) for thecurrent block may be derived based on the following equation.

$\begin{matrix}\left\{ \begin{matrix}{\left( {{mvx}_{0},{mvy}_{0}} \right) = \left( {{{mvpx}_{0} + {mvdx}_{0}},{{mvpy}_{0} + {mvdy}_{0}}} \right)} \\{\left( {{mvx}_{1},{mvy}_{1}} \right) = \left( {{{mvpx}_{1} - {mvdx}_{0}},{{mvpy}_{1} - {mvdy}_{0}}} \right)}\end{matrix} \right. & \left\lbrack {{Eq}.\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Equation 1, mvx₀ and mvy₀ may represent an x component and a ycomponent of L0 motion information or motion vector for L0 prediction,and mvx₁ and mvy₁ may represent an x component and a y component of L1motion information or motion vector for L1 prediction. Also, mvpx₀ andmvpy₀ may represent the x component and y component of the motion vectorpredictor for L0 prediction, and mvpx₁ and mvpy₁ may represent the xcomponent and y component of the motion vector predictor for L1prediction. Also, mvdx₀ and mvdy₀ may represent an x component and a ycomponent of a motion vector difference for L0 prediction.

Meanwhile, the MMVD mode is a method of applying motion vectordifference (MVD) to the merge mode, and motion information directly usedto generate prediction samples of the current block (i.e., the currentCU) may be implicitly derived. For example, an MMVD flag (i.e.,mmvd_flag) indicating whether to use MMVD for a current block (i.e., acurrent CU) may be signaled, and MMVD may be performed based on thisMMVD flag. When MMVD is applied to the current block (i.e., whenmmvd_flag is 1), additional information on MMVD may be signaled.

Here, the additional information on the MMVD includes a merge candidateflag (i.e., mmvd_cand_flag) indicating whether the first candidate orthe second candidate in the merge candidate list is used together withthe MVD, and a distance index (i.e., mmvd_distance_idx) for indicatingthe motion magnitude and a direction index (i.e., mmvd_direction_idx)for indicating a motion direction.

In the MMVD mode, two candidates located in the first and second entriesamong the candidates in the merge candidate list (i.e., the firstcandidate or the second candidate) may be used, and the two candidates(i.e., the first candidate or the second candidate) may be used. One ofthem may be used as the base MV. For example, a merge candidate flag(i.e., mmvd_cand_flag) may be signaled to indicate any one of twocandidates (i.e., the first candidate or the second candidate) in themerge candidate list.

Furthermore, distance index (i.e., mmvd_distance_idx) specifies motionmagnitude information and indicate the pre-defined offset from thestarting point. The offset may be added to either horizontal componentor vertical component of starting MV. The relation of distance index andpre-defined offset is specified in the following table.

TABLE 5 MmvdDistance[ x0 ][ y0 ] mmvd_distance_idx[ x0 ][ y0 ]tile_group_fpel_mmvd_enabled_flag = = 0tile_group_fpel_mmvd_enabled_flag = = 1 0 1 4 1 2 8 2 4 16 3 8 32 4 1664 5 32 128 6 64 256 7 128 512

Referring to Table 5 above, the distance of the MVD (eg, MmvdDistance)is determined according to the value of the distance index (eg,mmvd_distance_idx), and the distance of the MVD (eg, MmvdDistance) maybe derived by using integer sample precision or fractional sampleprecision based on the value of tile_group_fpel_mmvd_enabled_flag. Forexample, when tile_group_fpel_mmvd_enabled_flag is equal to 1, itindicates that the distance of the MVD is derived by using integersample precision in the current tile group (or picture header), and whentile_group_fpel_mmvd_enabled_flag is equal to 0, it indicates that thedistance of the MVD is derived by using fractional sample precision inthe tile group (or picture header). In Table 1, information (flag) for atile group may be replaced with information for a picture header, forexample, tile_group_fpel_mmvd_enabled_flag may be replaced withph_fpel_mmvd_enabled_flag (or ph_mmvd_fullpel_only_flag).

In addition, the direction index (eg, mmvd_direction_idx) indicates thedirection of the MVD with respect to the starting point, and mayindicate four directions as shown in Table 6 below. In this case, thedirection of the MVD may indicate the sign of the MVD. The relationshipbetween the direction index and the MVD sign may be expressed as thefollowing table.

TABLE 6 MmvdSign mmvd_direction idx[ x0 ][ y0 ] MmvdSign[ x0 ][ y0 ][0][ x0 ][ y0 ][1] 0 +1 0 1 −1 0 2 0 +1 3 0 −1

Referring to Table 6, the sign of the MVD (eg, MmvdSign) is determinedaccording to the value of the direction index (eg, mmvd_direction_idx),and the sign of the MVD (eg, MmvdSign) may be derived for the L0reference picture and the L1 reference picture.

Based on the above-described distance index (eg, mmvd_distance_idx) anddirection index (eg, mmvd_direction_idx), the offset of the MVD may becalculated using the following equations.MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]>>2)*MmvdSign[x0][y0][0]  [Eq.2]MmvdOffset[x0][y0][1]=(MnvdDistance[x0][y0]<2)*MmvdSign[x0][y0][1]  [Eq.3]

In Equations 2 and 3, the MMVD distance (MmvdDistance[x0] [y0]) and MMVDsigns (MmvdSign[x0][y0][0], MmvdSign[x0][y0][1]) may be derived based onTable 5 and/or Table 6. In summary, in the MMVD mode, a merge candidateindicated by a merge candidate flag (eg, mmvd_cand_flag) is selectedfrom among the merge candidates in the merge candidate list derivedbased on the neighboring blocks, and the selected merge candidate isused as a base candidate (eg, MVP). In addition, motion information (ie,motion vector) of the current block may be derived by adding an MVDderived using a distance index (eg, mmvd_distance_idx) and a directionindex (eg, mmvd_direction_idx) based on the base candidate.

A predicted block for the current block may be derived based on motioninformation derived according to the prediction mode. The predictedblock may include prediction samples (prediction sample array) of thecurrent block. When the motion vector of the current block indicates afractional sample unit, an interpolation procedure may be performed,through which prediction samples of the current block may be derivedbased on reference samples in the fractional sample unit within areference picture. When bi-prediction is applied, prediction samplesderived through weighting or weighted averaging (according to phase) ofprediction samples derived based on L0 prediction (that is, predictionusing reference picture and MVL0 in reference picture list L0) andprediction samples derived based on L1 prediction (that is, predictionusing reference picture and MVL1 in reference picture list L1) may beused as prediction samples of the current block. When bi-prediction isapplied, if the reference picture used for L0 prediction and thereference picture used for L1 prediction are located in differenttemporal directions with respect to the current picture (i.e.,bi-prediction and bidirectional prediction), it may be called truebi-prediction.

As described above, reconstructed samples and reconstructed pictures maybe generated based on the derived prediction samples, and thenprocedures such as in-loop filtering may be performed.

As described above, according to this document, when bi-prediction isapplied to the current block, prediction samples can be derived based ona weighted average. Conventionally, the bi-prediction signal (ie, thebi-prediction samples) can be derived through a simple average of the L0prediction signal (L0 prediction samples) and the L1 prediction signal(L1 prediction samples). That is, the bi-prediction samples were derivedas an average of the L0 prediction samples based on the L0 referencepicture and MVL0 and the L1 prediction samples based on the L1 referencepicture and MVL1. However, according to this document, whenbi-prediction is applied, a bi-prediction signal (bi-prediction samples)can be derived through a weighted average of the L0 prediction signaland the L1 prediction signal as follows.

In the above-described MMVD related embodiments, a method that considersa long-term reference picture in the MVD derivation process of MMVD maybe proposed, thereby maintaining and increasing compression efficiencyin various applications. In addition, the method proposed in theembodiments of this document can be equally applied to SMVD, which is asymmetric MVD technology used in inter mode (MVP mode), in addition tothe MMVD technology used in MERGE.

FIG. 8 is a diagram for describing a method of deriving motion vectorsin inter prediction.

In an embodiment of this document, it uses an MV derivation methodconsidering a long-term reference picture in a motion vector scalingprocess of a temporal motion candidate (a temporal motion candidate, atemporal merge candidate, or a temporal mvp candidate). The temporalmotion candidate may correspond to mvCol (mvLXCol). The temporal motioncandidate may be referred to as a TMVP.

The following table describes the definition of a long-term referencepicture.

TABLE 7 The function LongTermRefPic( aPic, aPb, refIdx, LX ), with Xbeing 0 or 1, is defined as follows: - If the picture with index refIdxfrom reference picture list LX  of the slice containing prediction blockaPb  in the picture aPic was marked as “used for long term reference” at the time when aPic was the current  picture, LongTermRefPic( aPic,aPb, refIdx, LX ) is equal to 1. - Otherwise, LongTermRefPic( aPic, aPb,refIdx, LX ) is equal to 0.

Referring to Table 7 above, if LongTermRefPic(aPic, aPb, refIdx, LX) isequal to 1 (true), the corresponding reference picture may be marked asused for long-term reference. For example, a reference picture notmarked as used for long-term reference may be a reference picture markedas used for short-term reference. In another example, a referencepicture not marked as used for long-term reference and not marked asunused may be a reference picture marked as used for short-termreference. Hereinafter, a reference picture marked as used for long-termreference may be referred to as a long-term reference picture, and areference picture marked as used for short-term reference may bereferred to as a short-term reference picture.

The following table describes the derivation of TMVP (mvLXCol).

TABLE 8 When availableFlagLXCol is equal to TRUE, mvLXCol andavailableFlagLXCol are derived as follows:  If LongTermRefPic( currPic,currCb, refldxLX, LX ) is not equal to LongTermRefPic( ColPic, colCb, refldxCol, listCol ), both components of mvLXCol are set equal to 0 andavailableFlagLXCol is set equal  to 0.  Otherwise, the variableavailableFlagLXCol is set equal to 1, refPicList┌ listCol ┐┌ refIdxCol ┐is set to be  the picture with reference index refIdxCol in thereference picture list listCol of the slice containing coding  blockcolCb in the collocated picture specified by ColPic, and the followingapplies:   colPocDiff = DiffPicOrderCnt( ColPic, refPicList[ listCol ][refldxCol ] ) (8-402)   currPocDiff = DiffPicOrderCnt( currPic,RefPicList[ X ][ refldxLX ] ) (8-403)   The temporal motion buffercompression process for collocated motion vectors as specified in  clause 8.5.2.15 is invoked with mvCol as input, and the modified mvColas output.   If RefPicListp[ X ][ refldxLX ] is a long-term referencepicture, or colPocDiff is equal to currPocDiff,   mvLXCol is derived asfollows:   mvLXCol = mvCol (8-404)   Otherwise, mvLXCol is derived as ascaled version of the motion vector mvCol as follows:   tx = ( 16384 + (Abs(td) td ) >> 1 ) )/td (8-405)   distScaleFactor = Clip3(−4096, 4095,( tb * tx + 32 ) >> 6 ) (8-406)   mvLXCol = Clip3( −131072, 131071,(distScaleFactor * mvCol +    128 − ( distScaleFactor * mvCol >= 0 )) >> 8 ) ) (8-407)   where td and tb are derived as follows:   td =Clip3( −128, 127, colPocDiff ) (8-408)   tb = Clip3( −128, 127,currPocDiff ) (8-409)

Referring to FIG. 8 and Table 8, when the type of the reference picturepointed to by the current picture (eg, indicating whether a long-termreference picture (LTRP) or a short-term reference picture (STRP)) isnot equal to the type of the collocated reference picture pointed to bythe collocated picture, the temporal motion vector mvLXCol is not used.That is, when all of them are the long-term reference pictures or theshort-term reference pictures, colMV is derived, otherwise, colMV is notderived. In addition, in the case that all of them are the long-termreference pictures and in the case where the POC difference between thecurrent picture and the reference picture of the current picture is thesame as the POC difference between the collocated picture and thereference picture of the collocated picture, the collocated motionvector can be used as it is without scaling. If it is a short-termreference picture and the POC difference is different, the motion vectorof the collocated block is used after scaled.

In an embodiment of this document, MMVD used in the MERGE/SKIP modesignals a base motion vector index (base MV index), a distance index,and a direction index for one coding block as information for derivingMVD information. In the case of unidirectional prediction, MVD isderived from motion information, and in the case of bidirectionalprediction, symmetric MVD information is generated using a mirroring andscaling method.

In the case of bidirectional prediction, MVD information for L0 or L1 isscaled to generate an MVD of L1 or L0. However, when a long-termreference picture is referred, it requires modification in the MVDderivation process.

FIG. 9 to 13 show MVD derivation methods of MMVD according toembodiments of the present document. The methods shown in FIGS. 9 to 13may be for a block to which bi-directional prediction is applied.

In one embodiment according to FIG. 9, when the distance to the L0reference picture and the distance to the L1 reference picture are thesame, the MmvdOffset derived can be used as the MVD as it is, and thePOC differences (POC difference between the L0 reference picture and thecurrent picture and the POC difference between the L1 reference pictureand the current picture) are different, MVD can be derived by scaling orsimple mirroring (ie, −1*MmvdOffset) according to the POC difference andwhether it is a long-term or short-term reference picture.

In one example, a method of deriving symmetric MVD using MMVD for ablock to which bi-prediction is applied is not suitable for a blockusing a long-term reference picture. It is difficult to expectperformance improvement. Accordingly, in the following drawings andembodiments, an example is introduced, in which MMVD is not applied whenreference picture types of L0 and L1 are different.

In one embodiment according to FIG. 10, a method for deriving MVD may bedifferent according to whether a reference picture referenced by acurrent picture (or a current slice, a current block) is a long-termreference picture (LTRP) or a short-term reference picture (STRP). Inone example, when the method of the embodiment according to FIG. 10 isapplied, a part of the standard document according to the presentembodiment may be described as shown in the following table.

TABLE 9 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process arc:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to thetop-left  luma sample of the current picture,  reference indicesrefIdxL0 and refIdxL1. prediction list utilization flags predFlagL0 andpredFlagL1 Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifics the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ])    currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL0] )   If currPocDiffL0 is equal to currPocDiffL1 and ,the followingapplies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = MmvdOffset[ xCb ][yCb ][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   Otherwise,if Abs( currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1), the following   applies:      td = Clip3( −128, 127, currPocDiffL0 )     tb = Clip3( −128, 127, currPocDiffL1 )      tx = ( 16384 + ( Abs(td ) >> 1 ) )/td      distScaleFactor = Clip3( −4096, 4095, ( tb * tx +32 ) >> 6 )      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]     If RefPicList[ 0 ][ refIdxL0 ] is not a long-term reference picture and RefPicList[ 1 ][refIdxL1 ]     is not a long-term reference picture, the followingapplies:      mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor *mMvdL0[ 0 ] +      128 − ( distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1] +      128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )    Otherwise, If RefPicList[ 0 ][ refIdxL0 ] is a long-term referencepicture and     RefPicList[ 1 ][ refIdxL1 ] is a long-term referencepicture, the following applies:      mMvdL1[ 0 ] = Sign( currPocDiffL0 )== Sign( currPocDiffL1 ) ?                mMvdL0[ 0 ]:−mMvdL0[ 0 ]     mMvdL0[ 0 ] = Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ?               mMvdL0[ 0 ]:−mMvdL0[ 0 ]      Otherwise, the followingapplies:       mMvdL0[ 0 ] = 0       mMvdL0[ 1 ] = 0       mMvdL1[ 0 ] =0       mMvdL1[ 1 ] = 0     Otherwise ( Abs( currPocDiffL0 ) is lessthan Abs( currPocDiffL1 )), the following applies:       td = Clip3(−128, 127, currPocDiffL1 )       tb = Clip3( −128, 127, currPocDiffL0 )      tx = ( 16384 + ( Abs( td ) >> 1 ) )/td       distScaleFactor =Clip3( −4096. 4095, ( tb * tx + 32 ) >> 6 )       mMvdL1[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb][ 1 ]      If RefPicList[ 0 ][ refIdxL0 ] is not a long-term referencepicture and RefPicList[ 1 ][ refIdxL1 ]      is not a long-termreference picture, the following applies:       mMvdL0[ 0 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] +        128 − (distScaleFactor * mMvdL1[ 0 ] >= 0 ) ) >> 8 )       mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1,, (distScaleFactor * mMvdL1[ 1 ] +        128 − (distScaleFactor * mMvdL1[ 1 ] >= 0 ) ) >> 8 ) )      Otherwise, IfRefPicList[ 0 ][ refidxL0 ] is a long-term reference picture and     RefPicList[ 1 ][ refIdxL1 ] is a long-term reference picture, thefollowing applies:       mMvdL0[ 0 ] = Sign( currPocDiffL0 ) == Sign(currPocDiffL1 ) ?                  mMvdL1[ 0 ]:−mMvdL1[ 0 ]      mMvdL0[ 1 ] = Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                 mMvdL1[ 1 ]:−mMvdL1[ 1 ]      Otherwise, the followingapplies:       mMvdL0[ 0 ] = 0       mMvdL0[ 1 ] = 0       mMvdL1[ 0 ] =0       mMvdL1[ 1 ] = 0  Otherwise ( predFlagL0 or predFlagL1 are equalto 1 ), the following applies for X being 0 and 1:       mMvdLX[ 0 ] = (predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ]:0       mMvdLX[ 1 ] =( predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ]:0

In one embodiment according to FIG. 11, a method for deriving MVD may bedifferent according to whether a reference picture referenced by acurrent picture (or a current slice, a current block) is a long-termreference picture (LTRP) or a short-term reference picture (STRP). Inone example, when the method of the embodiment according to FIG. 11 isapplied, a part of the standard document according to the presentembodiment may be described as shown in the following table.

TABLE 10 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process arc:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to thetop-left  luma sample of the current picture.  reference indicesrefIdxL0 and refIdxL1,  prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ])    currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL1] )   If (RefPicList[ 0 ][ refIdxL0 ] is a long-term reference pictureand RefPicList[ 1 ][ refIdxL1 ] is a   short-term reference picture) or(RefPicList[ 0 ][ refIdxL0 ] is a short-term reference picture and  RefPicList[ 1 ][ refIdxL1 ] is a long-term reference picture), thefollowing applies:      mMvdL0[ 0 ] = 0      mMvdL0[ 1 ] = 0     mMvdL1[ 0 ] = 0      mMvdL1[ 1 ] = 0   Otherwise, the followingapplies.     If currPocDiffL0 is equal to currPocDiffL1, the followingapplies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = MmvdOffset[ xCb ][yCb ][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]    Otherwise, if Abs( currPocDiffL0 ) is greater than or equal to Abs(currPocDiffL1 ) the     following applies:      td = Clip3( −128, 127,currPocDiffL0 )      tb = Clip3( −128, 127, currPocDiffL1 )      tx = (16384 + ( Abs( td ) >> 1 ) )/td      distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 )      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]      mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       IfRefPicList[ 0 ][ refIdxL0 ] is not a long-term reference picture, thefollowing applies:       mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 0 ] +       128 − ( distScaleFactor * mMvdL0[0 ] >= 0 ) ) >> 8 )       mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 1 ] +       128 − ( distScaleFactor * mMvdL0[1 ] >= 0 ) ) >> 8 )        Otherwise, the following applies:      mMvdL1[ 0 ] = Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ?               mMvdL0[ 0 ]:−mMvdL0[ 0 ]       mMvdL1[ 0 ] = Sign(currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                mMvdL0[ 0]:−mMvdL0[ 0 ]      Otherwise ( Abs( currPocDiffL0 ) is less than Abs(currPocDiffL1 )), the following applies:       td = Clip3( −128, 127,currPocDiffL1 )       tb = Clip3( −128, 127, currPocDiffL0 )       tx =( 16384 + ( Abs( td ) >> 1 ) )/td       distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 )       mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]        IfRefPicList[ 0 ][ refIdxL0 ] is not a long-term reference picture, thefollowing applies:       mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL1[ 0 ] +       128 − ( distScaleFactor * mMvdL1[0 ] >= 0 ) ) >> 8 )       mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,,(distScaleFactor * mMvdL1[ 1 ] +       128 − ( distScaleFactor * mMvdL1[1 ] >= 0 ) ) >> 8 ) )        Otherwise, the following applies:      mMvdL10[ 0 ] = Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                 mMvdL1[ 0 ]:−mMvdL1[ 0 ]       mMvdL0[ 1 ] = Sign(currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                  mMvdL1[ 1]:−mMvdL1[ 1 ]  Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ),the following applies for X being 0 and 1:       mMvdLX[ 0 ] = (predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ]:0       mMvdLX[ 1 ] =( predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ]:0

In summary, the MVD derivation process of MMVD, that does not deriveMVDs when the reference picture types in each direction are different,have been described.

In one embodiment according to FIG. 12, MVD may not be derived in allcases of referencing a long-term reference picture. That is, when atleast one L0 and L1 reference picture is a long-term reference picture,MVD is set to 0, and MVD can be derived only when a short-term referencepicture is included.

In one example, based on the highest priority condition (RefPicL0!=LTRP&& RefPicL1 !=STRP), MVD for MMVD may be derived when the currentpicture (or current slice, current block) refers to only short-termreference pictures. In one example, when the method of the embodimentaccording to FIG. 12 is applied, a part of the standard documentaccording to the present embodiment may be described as shown in thefollowing table.

TABLE 11 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of the topleft sample of the current luma coding block relatWe to the top-left luma sample of the current picture,  reference indices refIdxL0 andrefldxL1,  prediction list utilization flags predFlagL1 and predFlagL1.Outputs of this process are the luma merge motion vector differences in1/16 fractional-sample accuracy mMvdL0 and mMvdL1. The variable currPicspecifies the current picture. The luma merge motion vector differencesmMvdL0 and mMvdL1 are derived as follows:  If both predflagL0 andpredFlagL1 are equal to 1, the following applies:    currPocDiffL0 =DiffPicOrderCnt( currPic, RelPicList[ 0 ][ refldxL0 ] )    currPocDiffL1= DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refldxL1 ] )   IfRefPicList[ 0 ][ refIdxL0 ] is not a short-term reference picture orRefPicList[ 1 ][ refIdxL ] is not   a short-term reference picture, thefollowing applies:       mMvdL0[ 0 ] = 0       mMvdL0[ 1 ] = 0      mMvdL1[ 0 ] = 0       mMvdL1[ 1 ] = 0   Otherwise, the followingapplies:     If currPocDiffL0 isequal to currPocDiffL1, the followingapplies:       mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL0[1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0 ] = MmvdOffset[ xCb][ yCb ][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]    Otherwise, if Abs( currPocDifIL0 ) is greater than or equal to Abs(currPocDiffL1 ), the     following applies:       td = Clip3( −128, 127,currPocDiffL0 )       tb = Clip3( −128, 127, currPocDiffL1 )       tx =( 16384 + ( Abs( td ) >> 1 ) )/td       distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 )       mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]       mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0] = Clip3(−2¹⁵, 2¹⁵ − 1 (distScaleFactor * mMvdL0[ 0 ] +       128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] =Clip3(−2¹⁵, 2¹⁵ − 1 (distScaleFactor * mMvdL0[ 1 ] +     128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )    Otherwise (Abs(currPocDiffLO ) is less than Abs( currPocDiffL1 )), the followingapplies:     td = Clip3( −128, 127, currPocDiffL0 )     tb = Clip3(−128, 127, currPocDiffL1 )     tx = ( 16384 + ( Abs( td ) >> 1 ) )/td    distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32 ) >> 6 )    mMvdL1[ 0 ] = MmvdOffset┌ xCb ┐┌yCb ┐┌ 0 ┐     mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]     mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1(distScaleFactor * mMvdL1[ 0 ] +     128 − ( distScaleFactor * mMvdL1[ 0] >= 0) ) >> 8 )     mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1(distScaleFactor * mMvdL1[ 1 ] +     128 − ( distScaleFactor * mMvdL1[ 1] >= 0) ) >> 8 )  Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ),the following applies for X being 0 and 1:   If RefPicList[ X ][refIdxLX ] is not a short-tenn reference picture, the following applies:    mMvdLX[ 0 ] = 0     mMvdLX[ 1 ] = 0   Otherwise, the followingapplies     mMvdLX[ 0 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb][ 0 ]:0     mMvdLX[ 1 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb][ 0 ]:0

In one embodiment according to FIG. 13, when reference picture types ineach direction are different, MVD is derived when a short-term referencepicture is obtained, and MVD is derived to 0 when a long-term referencepicture is included.

In one example, when the reference picture types in each direction aredifferent, MmvdOffset is applied when referring to a reference picture(short-term reference picture) that is close to the current picture, andMVD has a value of 0 when referring to a reference picture (long-termreference picture) that is far from the current picture. In this case, apicture close to the current picture can be regarded as having ashort-term reference picture, but when the close picture is a long-termreference picture, mmvdOffset may be applied to a motion vector of alist indicating the short-term reference picture.

TABLE 12 mMvdL0 = 0 mMvdL1 = MmvdOffset mMvdL0 = 0 mMvdL1 = (−1) *MmvdOffset mMvdL0 = MmvdOffset mMvdL1 = 0 mMvdL0 = (−1) * MmvdOffsetmMvdL1 = 0

For example, the four paragraphs included in Table 12 may sequentiallyreplace the lowermost blocks (contents) of the flowchart included inFIG. 13.

In one example, when the method of the embodiment according to FIG. 13is applied, a part of the standard document according to the presentembodiment may be described as the following table.

TABLE 13 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to thetop-left  luma sample of the current picture,  reference indicesrefIdxL0 and refIdxL1  prediction list utilization flags predFlagL0 andpredFlagL1 Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, DefPicList[ 0 ][ refIdxL0 ])    currPocDiffL1 = DiffPicOrderCnt( currPic. RefFicList[ 1 ][ refIdxL1] )   if currPocDiffL0 is equal to currPocDiffL1 and, the followingapplies:       mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL0[1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0 ] = MmvdOffset[ xCb][ yCb ][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]  Otherwise, if Abs( currPocDiffL0 ) is greater than or equal to Abs(currPocDiffL1 ) the following   applies:       td = Clip3( −128, 127,currPocDiffL0 )       tb = Clip3( −128, 127, currPocDiffL1 )       tx =( 16384 + ( Abs( td ) >> 1 ) )/td       distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 )       mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]       mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]     IfRefPicList[ 0 ][ refIdxL0 ] is not a long-term reference picture andRefPicList[ 1 ][ refIdxL1 ]     is not a long-term reference picture.the following applies:       mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 0 ] +       128 − ( distScaleFactor * mMvdL0[0 ] >= 0 ) ) >> 8 )       mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 1 ] +       128 − ( distScaleFactor * mMvdL0[1 ] >= 0 ) ) >> 8 )     Otherwise, If RefPicList[ 0 ][ refIdxL0 ] is along-term reference picture and     RefPicList[ 1 ][ refIdxL1 ] is along-term reference picture, the following applies:       mMvdL1[ 0 ] =Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                mMvdL0[0 ]:−mMvdL0[ 0 ]       mMvdL1[ 1 ] = Sign( currPocDiffL0 ) == Sign(currPocDiffL1 ) ?                mMvdL0[ 0 ]:−mMvdL0[ 0 ]      Otherwise, the following applies:        mMvdL0[ 0 ] = 0       mMvdL0[ 1 ] = 0        mMvdL1[ 0 ] = MmvdOffset[ xCb ][yCb][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][yCb][ 1 ]    Otherwise ( Abs(currPocDiffL0 ) is less than Abs( currPocDiffL1 )), the followingapplies:        td = Clip3( −128, 127, currPocDiffL1 )        tb =Clip3( −128, 127, currPocDiffL0 )        tx = ( 16384 + ( Abs( td ) >> 1) )/td        distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32 ) >>6 )        mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]        mMvdL1[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]       If RefPicList[ 0 ][ refIdxL0 ]is not a long-term reference picture and RefPicList[ 1 ][ refIdxL1 ]      is not a long-term reference picture, the following applies:       mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScalcFactor * mMvdL1[ 0] +        128 − ( distScaleFactor * mMvdL1[ 0 ] >= 0 ) ) >> 8 )       mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,, (distScaleFactor * mMvdL1[ 1] +        128 − ( distScaleFactor * mMvdL1[ 1 ] >= 0 ) ) >> 8 ) )      Otherwise, If RefPicList[ 0 ][ refIdxL0 ] is a long-term referencepicture and       RefPicList[ 1 ][ refIdxL1 ] is a long-term referencepicture, the following applies:        mMvdL10[ 0 ] = Sign(currPocDiffL0 ) == Sign( currPocDiffL1 ) ?                  mMvdL1[ 0]:−mMvdL1[ 0 ]        mMvdL0[ 1 ] = Sign( currPocDiffL0 ) == Sign(currPocDiffL1 ) ?                  mMvdL1[ 1 ]:−mMvdL1[ 1 ]      Otherwise, the following applies:        mMydL0[ 0 ] = MmvdOffset[xCb ][ yCb ][ 0 ]        mMydL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       mMydL1[ 0 ] = 0        mMydL1[ 1 ] = 0  Otherwise ( predFlagL0 orpredFlagL1 are equal to 1 ), the following applies for X being 0 and 1:       mMvdLX[ 0 ] = ( predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0]:0        mMvdLX[ 1 ] = ( predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][1 ]:0

The following table shows a comparison table between the examplesincluded in this document.

TABLE 14 Embodiment A Embodiment B Embodiment C Embodiment D L0 L1 POCL0 Offset L1 Offset L0 Offset L1 Offset L0 Offset L1 Offset L0 Offset L1Offset Short Short Same Offset Offset Offset Offset Offset Offset OffsetOffset Diff (LO >= L1) Offset Scaled Offset Scaled Offset Scaled OffsetScaled Diff (LO < L1) Scaled Offset Scaled Offset Scaled Offset ScaledOffset Long Long Same Offset Offset Offset Offset 0 0 Offset Offset Diff(LO >= L1) Offset (−1) Offset Offset (−1) Offset 0 0 Offset (−1) OffsetDiff (LO < L1) (−1) Offset Offset (−1) Offset Offset 0 0 (−1) OffsetOffset Short Long Same N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO >= L1)N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO < L1) (−1) Offset Offset 0 0 00 Offset 0 Long Short Same N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO >=L1) Offset (−1) Offset 0 0 0 0 0 Offset Diff (LO < L1) N/A N/A N/A N/AN/A N/A N/A N/A

Referring to Table 14, a comparison between methods of applying anoffset in consideration of reference picture types for MVD derivation ofMMVD described in the embodiments according to FIG. 9 to FIG. 13 isshown. In Table 14, Embodiment A may relate to the existing MMVD,Embodiment B may show the embodiment according to FIG. 9 to FIG. 11,Embodiment C may show the embodiment according to FIG. 12, andEmbodiment D may show the embodiment according to FIG. 13.

That is, in the embodiment according to FIG. 9, FIG. 10, and FIG. 11,the method of deriving MVD only when the reference picture types of bothdirections are the same has been described, and in the embodimentaccording to FIG. 12, the method of deriving MVD only when bothdirections are short-term reference pictures has been described. In thecase of the embodiment according to FIG. 12, MVD may be set to 0 in thecase of a long-term reference picture for unidirectional prediction. Inaddition, in the embodiment according to FIG. 13, a method of derivingMVD in only one direction when reference picture types in bothdirections are different has been described. Differences between theembodiments represent various features of the techniques described inthis document, and it can be understood by those of ordinary skill inthe art that the effects to be achieved by the embodiments according tothis document can be implemented based on the features.

In an embodiment according to this document, when the reference picturetype is a long-term reference picture, a separate process is performed.When a long-term reference picture is included, the POC difference(POCDiff)-based scaling or mirroring does not affect performanceimprovement, so the MmvdOffset value is assigned to the MVD in thedirection having the short-term reference picture, and the value 0 isassigned to the MVD in the direction having the long-term referencepicture. In one example, when this embodiment is applied, a part of astandard document conforming to the present embodiment may be describedas shown in the following table.

TABLE 15 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of the topleft sample of the current luma coding block relative to the top left luma sample of the current picture.  reference indices refIdxL0 andrefIdxL1.  prediction list utilization flags predFlagL0 and predFlagL1.Outputs of this process are the luma merge motion vector differences in1/16 fractional-sample accuracy mMvdL0 and mMvdL1. The variable currPicspecifies the current picture. The luma merge motion vector differencesmMvdL0 and mMvdL1 are derived as follows:  If both predFlagL0 andpredFlagL1 are equal to 1, the following applies:   If RefPicList[ 0 ][refIdxL0 ] is a long-term reference picture or RefPicList[ 1 ][ refIdxL1] is a long-   term reference picture, the following applies.      IfRefPicList[ 0 ][ refIdxL0 ] is a short-term reference picture,      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0 ] = 0       mMvdL1[ 1 ] =0      Otherwise, if RefPicList[ 1 ][ refIdxL1] is a short-termreference picture,       mMvdL0 0 ] = 0       mMvdL0 1 ] = 0      mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]      Otherwise,       mMvdL0[ 0 ] = 0      mMvdL0[ 1 ] = 0       mMvdL1[ 0 ] = 0       mMvdL1[ 1 ] = 0  Otherwise, the following applies:    currPocDiffL0 = DiffPicOrderCnt(curr Pic, RefPicList[ 0 ][ refIdxL0 ] )    currPocDiffL1 =DiffPicOrderCnt( curr Pic, RefPicList[ 1 ][ refIdxL1 ] )     IfcurrPocDiffL0 is equal to currPocDiffL, the following applies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]       mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]      Otherwise,if Abs( currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1), the      following applies:       td = Clip3( −128, 127,currPocDiffL0 )       tb = Clip3( −128, 127, currPocDiffL1 )       tx =( 16384 + ( Abs( td ) >> 1 ) )/td       distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 )       mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]       mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL1[ 0] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 0 ] +       128 − (distScaleFactor * mMvdL1[ 0 ] >= 0 ) ) >> 8 )       mMvdL1[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] +       128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )      Otherwise ( Abs(currPocDiffL0 ) is less than Abs( currPocDiffL1 )), the followingapplies:       td = Clip3( −128, 127, currPocDifiL1 )       tb = Clip3(−128, 127, currPocDiffL0 )       tx = ( 16384 + ( Abs( Id ) >> 1 ) )/td      distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32 ) >> 6 )      mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]       mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]       mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL1[ 0 ] +       128 − (distScaleFactor * mMvdL1[0 ] >= 0 ) ) >> 8 )       mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,,(distScaleFactor * mMvdL1[ 1 ] +       128 − (distScaleFactor * mMvdL1[1 ] >= 0 ) ) >> 8 ) )  Otherwise ( predFlagL0 or predFlagL1 are equal to1 ), the following applies for X being 0 and 1:    If RefPicList[ X ][refIdxLX ] is a long-term reference picture      mMvdLX[ 0 ] = 0     mMvdLX[ 1 ] = 0    Otherwise,        mMvdLX[ 0 ] = ( predFlagLX ==1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ]:0        mMvdLX[ 1 ] = ( predFlagLX== 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ]:0

In another example, a portion of Table 15 may be replaced with thefollowing table. Referring to Table 16, Offset may be applied based on areference picture type other than POCDiff.

TABLE 16  - If both predFlagL0 and predFlagL1 are equal to 1,  thefollowing applies:   - If RefPicList[ 0 ][ refIdxL0 ] is a long-termreference picture or   RefPicList[ 1 ][ refIdxL1 ] is a long-termreference picture,   the following applies:   - If RefPicList[ 0 ][refIdxL0 ] is a short-term    reference picture,    mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][1 ]    mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise, if RefPicList[ 1 ][refIdxL1 ] is a short-term    reference picture,    mMvdL0 0 ] =−MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0 1 ] = −MmvdOffset[ xCb ][ yCb][ 1 ]    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise,     mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][1 ]    mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]

In another example, a portion of Table 15 may be replaced with thefollowing table. Referring to Table 17, it is possible to always setMmvdOffset to L0 and -MmvdOffset to L1 without considering the referencepicture type.

TABLE 17  - If both predFlagL0 and predFlagL1 are equal to 1,  thefollowing applies:   - If RefPicList[ 0 ][ refIdxL0 ] is a long-termreference picture or   RefPicList[ 1 ][ refIdxL1 ] is a long-termreference picture,   the following applies:    mMvdL0[ 0 ] = MmvdOffset[xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]

According to an embodiment of this document, SMVD in the inter mode maybe performed similarly to MMVD used in the above-described MERGE mode.In the case of bidirectional prediction, whether or not symmetric MVD isderived is signaled from the encoding apparatus to the decodingapparatus, and when the related flag (ex. sym_mvd_flag) is true (or thevalue is 1), The second direction MVD (eg, MVDL1) is derived throughmirroring of the first direction MVD (ex. MVDL0). In this case, scalingfor the first direction MVD may not be performed.

The following tables show syntaxes for a coding unit according to anembodiment of this document.

TABLE 18 Descriptor coding_unit( x0, y0, cbWidth, cbHeight, treeType ) { if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = = PRED_BI &&  !inter_affine_flag[ x0 ][ y0 ] && RefIldxSymL0 > −1 && RefIdxSymL1 >−1 )   sym_mvd_flag[ x0 ][ y0 ] ae(v)  if( inter_pred_idc[ x0 ][ y0 ] !=PRED_L1 ) {   if( NumRefIdxActive[ 0 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ])    ref_idx_10[ x0 ][ y0 ] ae(v)   mvd_coding( x0, y0, 0, 0 )   if(MotionModelIdc[ x0 ][ y0 ] > 0 )    mvd_coding( x0, y0, 0, 1 )  if(MotionModelIdc[ x0 ][ y0 ] > 1 )    mvd_coding( x0, y0, 0, 2 )  mvp_10_flag[ x0 ][ y0 ] ae(v)  } else {   MvdL0[ x0 ][ y0 ][ 0 ] = 0  MvdL0[ x0 ][ y0 ][ 1 ] = 0  }  if( inter_pred_idc[ x0 ][ y0 ] !=PRED_L0 ) {   if( NumRefIdxActive[ 1 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ])    ref_idx_11[ x0 ][ y0 ] ae(v)   if( mvd_11_zero_flag &&inter_pred_idc[ x0 ][ y0 ] = = PRED_B1 ) {    MvdL1[ x0 ][ y0 ][ 0 ] = 0   MvdL1 [ x0 ][ y0 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ] = 0   MvdCpL1[ x0 ][ y0 ][ 0 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 1 ][ 0 ] =0    MvdCpL1[ x0 ][ y0 ][ 1 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 0 ]= 0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 1 ] = 0   } else {    if( sym_mvd_flag[x0 ][ y0 ] ) {     MvdL1[ x0 ][ y0 ][ 0 ] = −MvdL0[ x0 ][ y0 ][ 0 ]    MvdL1[ x0 ][ y0 ][ 1 ] = −MvdL0[ x0 ][ y0 ][ 1 ]    } else    mvd_coding( x0, y0, 1, 0 )    if( MotionModelIdc[ x0 ][ y0 ] > 0 )    mvd_coding( x0, y0, 1, 1 )    if(MotionModelIdc[ x0 ][ y0 ] > 1 )    mvd_coding( x0, y0, 1, 2 )    mvp_11_flag[ x0 ][ y0 ] ae(v)   }  }else {   MvdL1[ x0 ][ y0 ][ 0 ] = 0   MvdL1[ x0 ][ y0 ][ 1 ] = 0  } . .. }

TABLE 19 if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = =PRED_BI &&  !inter_affine_flag[ x0 ][ y0 ] &&  RefIdxSymL0 > −1 &&RefIdxSymL1 > −1 )  sym_mvd_flag[ x0 ][ y0 ] ae(v)

Referring to Tables 18 and 19, when inter_pred_idc==PRED_BI andreference pictures of L0 and L1 are available (eg, RefIdxSymL0>−1 &&RefIdxSymL1>−1), sym_mvd_flag is signaled.

The following table shows a decoding procedure for MMVD referenceindices according to an example.

TABLE 20 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences. i.e., When sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] −1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] )                     <    DiffPicOrderCnt( currPic, RefPicList[ 0 ][RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equalto −1.  For each index i with i = 0..NumRefIdxActive[ 1 ] −1, thefollowing applies:   When all of the following conditions are true,RefIdxSymL1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i] ) > 0,    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] )                     >    DiffPicOrderCnt( currPic, RefPicList[ 1 ][RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.  When RefIdxSymL0 isequal to −1 or RefIdxSymL1 is equal to −1, the following applies:   Foreach index i with i = 0..NumRefIdxActive[ 0 ] −1, the following applies:   When all of the following conditions are true, RefIdxSymL0 is set toi:     DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] )                    >     DiffPicOrderCnt( currPic, RefPicList[ 0 ][RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1.   For each index i with i= 0..NumRefIdxActive[ 1 ] −1, the following applies:    When all of thefollowing conditions are true, RefIdxSymLI is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0,    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] )                    <     DiffPicOrderCnt( currPic, RefPicList[ 1 ][RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.

Referring to Table 20, a procedure for deriving availability ofreference pictures of L0 and L1 is described. That is, if there is areference picture in the forward direction among the L0 referencepictures, the index of the reference picture closest to the currentpicture is set as RefIdxSynmL0, and the corresponding value is set asthe reference index of L0. In addition, when if there is a referencepicture in the backward direction among the L1 reference pictures, theindex of the reference picture closest to the current picture is set asRefIdxSymL1, and the corresponding value is set as the reference indexof L1.

Table 21 below shows a decoding procedure for MMVD reference indicesaccording to another example.

TABLE 21 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of fins process are RefIdxSymL0 and RefIdxSym1specifying the list 0 and list 1 reference picture indices for symmetricmotion sector differences, i.e.. when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive [ 0 ] −1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equal to −1.  For eachindex i with 0..NumRefIdxActive[ 1 ] −1, the following applies:   Whenall of the following conditions are true, RefIdxSymL1 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0,   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) <   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i = 0 ,NumRefIdxActive[ 0 ] − 1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) >    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1.   For each index i with i =0..NumRefIdxActive[ 1 ] −1, the following applies:    When all of thefollowing conditions are true, RefIdxSymLl is set to i.    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0,    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1  When RefIdxSymL0 is not equal to −1 andRefIdxSymL1 is not equal to −1, the following applies   If (RefPicList[0 ][ RefIdxSymL0 ] is a long-term reference picture and RefPicList[ I ][RefI   dxSymL1 ] is a short-term reference picture) or (RefPicList[ 0 ][RefIdxSymL0 ] is a short-t   erm reference picture and RefPicList[ 1 ][RefIdxSymL1 ] is a long term reference picture),   RefIdxSymL0 andRefIdxSymL1 are set to −1.

Referring to Table 21, as in the embodiment described with FIG. 9, FIG.10 and FIG. 11 when the types of L0 or L1 reference pictures aredifferent, that is, if the reference picture types of L0 and L1 aredifferent after reference index derivation for SMVD, SMVD is not used inorder to prevent SMVD in a case that the long term reference picture andthe short term reference picture are used (refer to the lowermostparagraph of Table 20).

In an embodiment of this document, SMVD may be applied in the inter modesimilar to MMVD used in the merge mode. When a long-term referencepicture is used as in the embodiment described with FIG. 12 thelong-term reference picture may be excluded from the reference indexderivation process for SMVD as shown in the following table in order toprevent SMVD.

TABLE 22 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0.   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1,    RefPicList[ 0 ][ i ] is ashort-term-reference picture.  RefidxSymL1 is set equal to −1  For eachindex i with i = 0..NumRefIdxActive[ 1 ] − 1, the following applies:  When all of the following conditions are true, RefIdxSymL1 is set toi:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0,   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1,    RefPicList[ 1][ i ] is ashort-term-reference picture.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i = 0..NumRefIdxActive[ 0 ] −1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicLisf[ 0 ][ i ] ) < 0,    DiffPicOrderCnt( currPic, RefPicLisf[ 0 ][ i ] ) >    DiffPicOrderCnt( currPic, RefPicLisf[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1,     RefPicList[ 0 ][ i ] is ashort-term-reference picture.   For each index i with i =0..NumRefIdxActive[ 1 ] −1, the following applies:    When all of thefollowing conditions are true, RefIdxSymL1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0.    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1,     RefPicList[ 1 ][ i ] is ashort-term-reference picture.

The following table according to another example of this embodimentshows an example of processing not to apply SMVD when a long-termreference picture is used after reference picture index derivation forSMVD.

TABLE 23 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equal to −1.  For eachindex i with i = 0..NuRefIdxActive[ 1 ] − 1, the following applies:  When all of the following conditions are true, RefIdxSymLl is set toi:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0.   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i 0..NumRefIdxActive 0 ][ − 1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) >    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1.   For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:    When all of thefollowing conditions are true. RefIdxSym1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ]) > 0,    DiffPicOrderCnt( currPic, RetPicList[ 1 ][ i ]) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1  When RefIdxSymL0 is not equal to −1 andRefIdxSymL1 is not equal to −1, the following applies   If RefPicList[ 0][ RefIdxSymL0 ] is a long-term reference picture or RefPicList[ 1 ][RefIdx   SymL1 ] is a long-term reference picture, RefIdxSymL0 andRefIdxSymL1 are set to −1.

In one embodiment of this document, in the colMV derivation process ofTMVP, when the reference picture type of the current picture and thereference picture type of the collocated picture are different, themotion vector MV is set to 0, but the derivation method in the case ofMMVD and SMVD are not same as for TMVP and they need to be unified.

Even when the reference picture type of the current picture is along-term reference picture and the reference picture type of thecollocated picture is a long-term reference picture, the motion vectoruses the value of the collocated motion vector as it is, but MV may beset to 0 in MMVD and SMVD. In this case, TMVP also sets MV to 0 withoutadditional induction.

In addition, even if the reference picture types are different, along-term reference picture having a close distance to the currentpicture may exist. Therefore, instead of setting the MV to 0, colMV maybe used as the MV without scaling.

FIG. 14 is a diagram for describing symmetric motion vector differences(SMVD) according to one embodiment of the present disclosure.

A method as shown in FIG. 15 may be used for deriving SMVD. In otherwords, SMVD may be derived based on a short-term reference picture(STRP) and/or a long-term reference picture (LTRP). If different typesof reference pictures are used when a mirrored L0 MVD is used as an L1MVD, an inaccurate MVD may be derived. It is so because a distance ratio(distance between reference picture 0 and a current picture and distancebetween reference picture 1 and the current picture) increases, and thecorrelation between motion vectors at different directions decreases.

According to one embodiment of the present disclosure, availability ofreference pictures is checked, and if conditions are met, sym_mvd_flagmay be parsed. If sym_mvd_flag is true, MVD of L1 (MVDL1) may be derivedas a mirrored MVDL0 (MVD of L0).

The following table shows part of coding unit syntax according to thepresent embodiment.

TABLE 24  if( sps_sinvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = =PRED_BI &&   !inter_affine_flag[ x0 ][ y0 ] && RefIdxSymL0 > −1 &&RefIdxSymL1 > −1 )   sym_mvd_flag[ x0 ][ y0 ] ae(v) if( inter_pred_idc[x0 ][ y0 ] != PRED_L1 ) {   if( NumRefIdxActive[ 0 ] > 1 &&!sym_mvd_flag[ x0 ][ y0 ] )    ref_idx_l0[ x0 ][ y0 ] ac(v)  mvd_coding( x0, y0, 0, 0 )   if( MotionMoclelIde[ x0 ][ y0 ] > 0 )   mvd_coding( x0, y0, 0 1 )   if(MotionModelIdc[ x0 ][ y0 ] > 1 )   mvd_coding( x0, y0, 0, 2 )   mvp_l0_flag[ x0 ][ y0 ] ae(v)  } else {  MvdL0[ x0 ][ y0 ][ 0 ] = 0   MvdL0[ x0 ][ y0 ][ 1 ] = 0  }  if(inter_pred_idc[ x0 ][ y0 ] != PRED_L0 ) {   if( NuniRefIdxActive[ 1 ] >1 && !sym_mvd_flag[ x0 ][ y0 ] )    ref_idx_l1[ x0 ][ y0 ] ae(v)   if(mvd_l1_zero_flag && inter_pred_idc[ x0 ][ y0 ] = = PRED_BI ) {   MvdL1[x0 ][ y0 ][ 0 ] = 0    MvdL1[ x0 ][ y0 ][ 1 ] = 0    MvdCpL1[x0 ][ y0 ][ 0 ][ 0 ] = 0    MvdCpL1[ x0 ][ y0 ][ 0 ][ 1 ] = 0   MvdCpL1[ x0 ][ y0 ][ 1 ][ 0 ] = 0    MvdCpL1[ x0 ][ y0 ][ 1 ][ 1 ] =0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 0 ] = 0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 1 ]= 0   } else {    if( syrn_mvd_flag[ x0 ][ y0 ] ) {     MvdL1[ x0 ][ y0][ 0 ] = −MvdL0[ x0 ][ y0 ][ 0 ]     MvdL1[ x0 ][ y0 ][ 1 ] = −MvdL0[ x0][ y0 ][ 1 ]    } else     mvd_coding( x0, y0, 1, 0 )    if(MotionModelIdc[ x0 ][ y0 ] > 0 )     mvd_coding( x0, y0, 1, 1 )   if(MotionModelIdc[ x0 ][ y0 ] > 1 )     mvd_coding( x0, y0, 1, 2 )   mvp_l1_falg[ x0 ][ y0 ] ae(v)   }  } else {   MvdLl[ x0 ][ y0 ][ 0 ]= 0   MvdLl[ x0 ][ y0 ][ 1 ] = 0  }

A procedure for deriving sym_mvd_flag according to the presentembodiment may be described based on Table 24.

In the present embodiment, reference picture indices for SMVD(RefIdxSymLX with X=0, 1) may be derived. RefIdxSymL0 may represent (theindex of) the closest reference picture having a POC smaller than thatof the current picture. RefIdxSymL1 may represent (the index of) theclosest reference picture having a POC larger than that of the currentpicture.

The following table describes a method for deriving reference pictureindices for SMVD according to the present embodiment in the form of astandard document.

TABLE 25 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equal to −1.  For eachindex i with i = 0..NuRefIdxActive[ 1 ] − 1, the following applies:  When all of the following conditions are true, RefIdxSymLl is set toi:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0.   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i 0..NumRefIdxActive 0 ][ − 1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) >    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1.   For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:    When all of thefollowing conditions are true. RefIdxSym1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ]) > 0,    DiffPicOrderCnt( currPic, RetPicList[ 1 ][ i ]) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1.

The following table shows a comparison result between embodiments. Byconsidering the reference picture type according to the embodimentsincluded in Table 26, the MVD accuracy of SMVD may be improved. In Table26, MVD may represent MVD 0 (MVD of L0).

TABLE 26 Embodiment P Embodiment Q Embodiment R L0 L1 L0 MVD L1 MVD L0MVD L1 MVD L0 MVD L1 MVD short Short MVD (−1)MVD MVD (−1)MVD MVD (−1)MVDlong Long MVD (−1)MVD MVD (−1)MVD short Long MVD (−1)MVD bong short MVD(−1)MVD

Referring to Table 26, embodiment P describes an existing method forderiving SMVD. In embodiment Q, SMVD may be limited when mixed referencepicture types (for example, STRP/LTRP or LTRP/STRP) are used in L0 andL1. In embodiment R, SMVD may be limited when a long-term referencepicture (LTRP) is referenced.

The following table describes a method for deriving reference pictureindices for SMVD according to the embodiment Q of Table 26 in the formof a standard document.

TABLE 27 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equal to −1.  For eachindex i with i = 0..NuRefIdxActive[ 1 ] − 1, the following applies:  When all of the following conditions are true, RefIdxSymLl is set toi:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0.   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i 0..NumRefIdxActive 0 ][ − 1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) >    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1.   For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:    When all of thefollowing conditions are true. RefIdxSym1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ]) > 0,    DiffPicOrderCnt( currPic, RetPicList[ 1 ][ i ]) ) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1.  When RefIdxSymL0 is not equal to −1 andRefIdxSymL1 is not equal to −1, the following applies   If (RefPicList[0 ][ RefIdxSymL0 ] is a long-term reference picture and   RefPicList[ 1][ RefIdxSymL1 ] is a short term reference picture) or   (RefPicList[ 0][ RefIdxSymL0 ] is a short-term reference picture and   RefPicList[ 1][ RefIdxSymL1 ] is a long-term reference picture), RefIdxSymL0 andRefIdxSymL1   are set equal to −1.

The following tables describe methods for deriving reference pictureindices for SMVD according to the embodiment Q of Table 26 in the formof a standard document.

TABLE 28 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.    RefPicList[ 0 ][ i ] is short-termreference picture.  RefIdxSymL1 is set equal to −1.  For each index iwith i = 0..NuRefIdxActive[ 1 ] − 1, the following applies:   When allof the following conditions are true, RefIdxSymLl is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0.   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.    RefPicList[ 0 ][ i ] is short-termreference picture.  When RefIdxSymL0 is equal to −1 or RefIdxSymL1 isequal to −1, the following applies:   For each index i with i0..NumRefIdxActive 0 ][ − 1, the following applies:    When all of thefollowing conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) >   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.    RefPicList[ 0 ][ i ] is short-termreference picture.   For each index i with i = 0..NumRefIdxActive[ 1 ] −1, the following applies:    When all of the following conditions aretrue. RefIdxSym1 is set to i:     DiffPicOrderCnt( currPic, RefPicList[1 ][ i ]) > 0,    DiffPicOrderCnt( currPic, RetPicList[ 1 ][ i ]) ) <   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.    RefPicList[ 0 ][ i ] is short-termreference picture.

TABLE 29 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows:  The variable currPic specifies the current picture. RefIdxSymL0 is set equal to −1.  For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:   When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  RefIdxSymL1 is set equal to −1.  For eachindex i with i = 0..NuRefIdxActive[ 1 ] − 1, the following applies:  When all of the following conditions are true, RefIdxSymLl is set toi:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0.   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) ) >   DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.  When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies:   For each index iwith i 0..NumRefIdxActive 0 ][ − 1, the following applies:    When allof the following conditions are true, RefIdxSymL0 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) ) >    DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to     −1.   For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:    When all of thefollowing conditions are true. RefIdxSym1 is set to i:    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ]) > 0,    DiffPicOrderCnt( currPic, RetPicList[ 1 ][ i ]) <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to     −1.  When RefIdxSymL0 is not equal to −1 andRefIdxSymL1 is not equal to −1, the following applies:   If (RefPicList[0 ][ RefIdxSymL0 ] is a long-term reference picture or   RefPicList[ 1][ RefIdxSymL1 ] is a long-term reference picture, RefIdxSymL0 andRefIdxSymL1   are set to −1.

Referring to Table 28 and/or Table 29, SMVD may be limited when along-term reference picture (LTRP) is referenced. For example, referringto Table 28, a long-term reference picture may be excluded from areference picture checking process. Accordingly, other referencepictures (rather than a long-term reference picture, for example) may beconsidered for SMVD. Referring to Table 29, when a reference pictureclosest to the current picture is a long-term reference picture, SMVDmay not be performed. For example, even if short-term reference picturesare included in a reference picture list, SMVD may not be performed whenthe closest reference picture to the current picture is the long-termreference picture.

In an example according to one embodiment of the present disclosure,when the POC distance of L0 is greater than or equal to the POC of L1 inthe MMVD procedure, L1 MVD may be derived as a scaled or mirrored L0MVD. When the POC distance of L0 is smaller than the POC of L1 in theMMVD procedure, L0 MVD may be derived as a scaled or mirrored L1 MVD inthe MMVD procedure.

FIG. 15 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure.

In one embodiment of the present disclosure, an MVD may be derived froman MMVD by considering POC differences and/or reference picture types.Referring to FIG. 15, currPocDiffLX may represent a difference betweenthe POC of a current picture and the POC of a reference picture LX.currPocDiffL0 and currPocDiffL1 may be compared with each other, and thetypes of reference pictures may be checked (“refPicList0!=LTRP” or“refPicList1!=LTRP”). Considering the conditions, MmvdOffset(derivedusing mmvd_cand_flag, mmvd_distance_idx, and/or mmvd_direction_idx) maybe allocated as a value the same as mMvdLX, a mirrored or scaled valueof mMvdLX.

The following table represents a part of the standard document accordingto the present embodiment.

TABLE 30 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative  to thetop-left luma sample of the current picture,  reference indices refIdxL0and refIdxL1,  prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refldxL0 ]) (8-336)    currPocDiffL1 = DiffPicOrderCnt( currPic, RetPicList[ 1 ][refldxL1 ] ) (8-337)   If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339)    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)     mMvdL1[ 1 ]= MmvdOffset[ xCb ][ yCb ][ 1 ] (8-341)   Otherwise, if Abs(currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1 ),   thefollowing applies:     td = Clip3(−128, 127, currPocDiffL0 )                  (8-342)     td = Clip3(−128, 127, currPocDiffL1 )                  (8-343)     tx − (16384 + ( Abs( td ) >> 1 ) ) / td                  (8-344)     distScaleFactor = Clip3( −4096, 4095, (tb * tx + 32 ) >> 6 ) (8-345)     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ] (8-346)     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-347)   If RefPicList[ 0 ][ refIdxL0 ] is not a long-term reference pictureand RefPicList[ 1 ]    [ refIdxL1 ] is not a long-term referencepicture, the following applies:     mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 0 ] + (8-348)                   128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] + (8-349)                  128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )   Otherwise, the following applies:     mMvdL1[ 0 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffLl ) ?              mMvdL0[ 0 ] :−mMvdL0[ 0 ] (8-350)     mMvdL1[ 1 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffLl ) ?              mMvdL0[ 1 ] : −mMvdL0[ 1 ] (8-351)   Otherwise (Abs( currPocDiffL0 ) is less than Abs( currPocDiffL1 )),the following applies:     td − Clip3( −128, 127, currPocDiffL1 )                  (8-352)     tb = Clip3( −128, 127, currPocDiffL0 )                  (8-353)     tx = ( 16384 + ( Abs( td) >> 1 ) ) / td                  (8-354)     distScaleFactor = Clip3( −4096_ 4095,(tb * tx + 32) >> 6) (8-355)     mMvdLl[ 0 ] = MmvdOffset[ xCb ][ yCb ][0 ] (8-356)     mMvdLl[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-357)   If RefPicList[ 0 ][ refIdxL0 ] is not a long-term reference pictureand RefPicList[ 1]    [ refIdxL1 ] is not a long-term reference picture,the following applies:     mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL1[ 0 ] + (8-358)                  128 − (distScaleFactor * mMvdL1[ 0 ] >= 0 ) ) >> 8 )     mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 1 ] + (8-359)                 128 − ( distScaleFactor * mMvdL1[ 1 ] >= 0 ) ) >> 8 )   Otherwise, the following applies:     mMvdL10[ 0 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffLl ) ?              mMvdL1[ 0 ] :−mMvdL1[ 0 ] (8-360)     mMvdL0[ 1 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffLl ) ?              mMvdL1[ 1 ] : −mMvdL1[ 1 ] (8-361)   Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ), the followingapplies for    X being 0 and 1:     mMvdLX[ 0 ] = ( predFlagLX = = 1) ?MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-362)     mMvdLX[ 1 ] = ( predFlagLX= = 1) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0 (8-363)

When the current picture refers to one or more long-term referencepictures (LTRPs), a mirroring procedure that considers a POC distancemay not be required. It is so because a mirroring MVD obtained from areference picture at a very far distance than other MVDs is noteffective in terms of accuracy. Methods for solving the problem abovewill be described below.

The following table represents a comparison result between embodiments.

TABLE 31 Embodiment X Embodiment Y Embodiment Z L0 L1 POC L0 Offset L1Offset L0 Offset L1 Offset L0 Offset L1 Offset Short Short Same OffsetOffset Offset Offset Offset Offset Diff (L0 >= L1) Offset Scaled OffsetScaled Offset Scaled Diff (L0 < L1) Scaled Offset Offset Offset OffsetScaled Long Long Same Offset Offset Offset Offset Offset Offset Diff(L0 >= L1) Offset (−1) Offset Offset (−1) Offset Offset (−1) Offset Diff(L0 < L1) (−1) Offset Offset Offset (−1) Offset Offset (−1) Offset ShortLong Same N/A N/A N/A N/A N/A N/A Diff (L0 >= L1) Offset (−1) OffsetOffset (−1) Offset Offset (−1) Offset Diff (L0 < L1) (−1) Offset OffsetOffset (−1) Offset Offset (−1) Offset Long Short Same N/A N/A N/A N/AN/A N/A Diff (L0 >= L1) Offset (−1) Offset Offset (−1) Offset Offset(−1) Offset Diff (L0 < L1) (−1) Offset Offset Offset (−1) Offset Offset(−1) Offset

Referring to Table 31, embodiment X shows an existing method forderiving an MMVD. In embodiment Y, the MMVD procedure may be limitedwhen the current block references one or more long-term referencepictures. In other words, a procedure for comparing POC distances for along-term reference picture may be omitted in embodiment Y In embodimentZ, the procedure for deriving MMVD may be limited for all cases. Inother words, the procedure for comparing POC distances for all cases maybe omitted in embodiment Z. In Table 29, Offset may refer to MmvdOffset.

FIG. 16 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure. The flow diagramof FIG. 16 may describe a method for deriving an MMVD according toembodiment Y above.

Referring to FIG. 16, when a reference picture type is a long-termreference picture, conditions for comparing POC differences may beremoved, and an anchor MVD used for a mirroring procedure may be fixedto L0 MVD.

The following table describes a method for deriving MMVDs according toembodiment Y of Table 31 in the form of a standard document.

TABLE 32 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top- left luma sample of the current picture,  reference indices refIdxL0and refIdxL1,  prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refldxL0 ]) (8-336)    currPocDiffL1 = DiffPicOrderCnt( currPic, RetPicList[ 1 ][refldxL1 ] ) (8-337)   If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339)    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)     mMvdL1[ 1 ]= MmvdOffset[ xCb ][ yCb ][ 1 ] (8-341)   If RefPicList[ 0 ][ refIdxL0 ]is long-term reference picture or RefPicList[ 1 ][ refIdxL1 ) is a  long-term reference picture, the following applies     mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-342)     mMvdL0[ 1 ] = MmvdOffset[ xCb][ yCb ][ 1 ] (8-343)     mMvdL1[ 0 ] = −mMvdL0[ 0 ]     (8-344)    mMvdL1[ 1 ] = −mMvdL0[ 1 ]     (8-345)   Otherwise, the followingapplies :     td = Clip3(−128, 127, currPocDiffL0 ) (8-346)     tb =Clip3(−128, 127, currPocDiffL1 ) (8-347)     tx − (16384 + ( Abs( td) >> 1 ) ) / td (8-348)     distScaleFactor = Clip3( −4096, 4095, ( tb *tx + 32 ) >> 6 ) (8-349)    If Abs( currPocDiffL0 ) is greater than orequal to Abs( currPocDiffL1 ), the following applies:     mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-350)     mMvdL0[ 1 ] = MmvdOffset[ xCb][ yCb ][ 1 ] (8-351)     mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 0 ] + (8-352)                  128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] + (8-353)                 128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )   Otherwise (Abs( currPocDiffL0 ) is less than or equal to Abs(currPocDiffL1 )),    the following applies:     mMvdL1[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-354)     mMvdL1[ 1 ] = MmvdOffset[ xCb][ yCb ][ 1 ] (8-355)     mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL1[ 0 ] + (8-356)                  128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 1 ] + (8-357)                 128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )   Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ), the followingapplies for X being 0 and 1:     mMvdLX[ 0 ] = ( predFlagLX = = 1) ?MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-368)     mMvdLX[ 1 ] = ( predFlagLX= = 1) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0 (8-369)

FIG. 17 is a flow diagram illustrating a method for deriving an MMVDaccording to one embodiment of the present disclosure. The flow diagramof FIG. 17 may describe a method for deriving an MMVD according toembodiment Z above.

Referring to FIG. 17, a procedure for deriving an MMVD may be limitedfor all cases. Conditions for comparing POC differences may be removedfor all cases, and an anchor MVD used for a mirroring or scalingprocedure may be fixed to L0 MVD.

The following table describes a method for deriving MMVDs according toembodiment Z of Table 31 in the form of a standard document.

TABLE 33 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative  to thetop-left luma sample of the current picture,  reference indices refIdxL0and refIdxL1,  prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refldxL0 ]) (8-336)    currPocDiffL1 = DiffPicOrderCnt( currPic, RetPicList[ 1 ][refldxL1 ] ) (8-337)   If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339)    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)     mMvdL1[ 1 ]= MmvdOffset[ xCb ][ yCb ][ 1 ] (8-341)   If RefPicList[ 0 ][ refIdxL0 ]is long-term reference picture or RefPicList[ 1 ][ refIdxL1 ]   is not along-term reference picture, the following applies:     mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-342)     mMvdL0[ 1 ] = MmvdOffset[ xCb][ yCb ][ 1 ] (8-343)     mMvdL1[ 0 ] = −mMvdL0[ 0 ] (8-344)     mMvdL1[1 ] = −mMvdL0[ 1 ] (8-345)   Otherwise, the following applies :     td =Clip3(−128, 127, currPocDiffL0 ) (8-346)     tb = Clip3(−128, 127,currPocDiffL1 ) (8-347)     tx − (16384 + ( Abs( td ) >> 1 ) ) / td(8-348)     distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32 ) >> 6) (8-349)     mMvdL0[ 0 ] − MmvdOffset[ xCb ][ yCb ][ 0 ] (8-350)    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-351)     mMvdL1[ 0 ]= Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 0 ] + (8-352)                 128 − ( distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )    mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] +(8-353)                  128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 )) >> 8 )    Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ), thefollowing applies for    X being 0 and 1:     mMvdLX[ 0 ] = ( predFlagLX= = 1) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-354)     mMvdLX[ 1 ] = (predFlagLX = = 1) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0 (8-355)

Also, in one example of the present embodiment, conditions for comparingPOC differences may be removed for all cases, and only the mirroringprocedure may be used. The following table describes a method forderiving MMVDs according to the present embodiment in the form of astandard document.

TABLE 34 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are:  a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative   to thetop-left luma sample of the current picture,  reference indices refIdxL0and refIdxL1,  prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows:  If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refldxL0 ]) (8-336)    currPocDiffL1 = DiffPicOrderCnt( currPic, RetPicList[ 1 ][refldxL1 ] ) (8-337)   If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339)    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)     mMvdL1[ 1 ]= MmvdOffset[ xCb ][ yCb ][ 1 ] (8-341)   Otherwise, the followingapplies :     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-346)    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-347)     mMvdL1[ 0 ]= −mMvdL0[ 0 ] (8-348)     mMvdL1[ 1 ] = −mMvdL0[ 1 ] (8-349)   Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ), the followingapplies for    X being 0 and 1:     mMvdLX[ 0 ] = ( predFlagLX = = 1) ?MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-358)     mMvdLX[ 1 ] = ( predFlagLX= = 1) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0 (8-359)

The following drawings are created to explain a specific example of thepresent specification. Since the names of specific devices described inthe drawings or the names of specific signals/messages/fields arepresented by way of example, the technical features of the presentspecification are not limited to the specific names used in thefollowing drawings.

FIGS. 18 and 19 illustrate a video/image encoding method and one exampleof a related component according to an embodiment(s) of the presentdisclosure. The encoding apparatus of FIG. 2 may perform the method ofFIG. 18. Specifically, for example, the predictor 220 of the encodingapparatus may perform the S1800 to S1850 steps of FIG. 18, and theresidual processor 230 of the encoding apparatus may perform the S1860step. The entropy encoder 240 of the encoding apparatus may perform theS1870 step. The method of FIG. 18 may include the embodiments of thepresent disclosure described above.

Referring to FIG. 18, the encoding apparatus derives an inter predictionmode for a current block within a current picture S1800. Here, the interprediction mode may include the merge mode, AMVP mode (the mode usingmotion vector predictor candidates), MMVD, and SMVD.

The encoding apparatus may derive reference pictures for the interprediction mode. The encoding apparatus may construct reference picturelists for deriving reference pictures. In one embodiment, referencepicture lists may include reference picture list 0 (or L0, referencepicture list L0) or reference picture list 1 (or L1, reference picturelist L1). For example, the encoding apparatus may construct referencepicture lists for each slice included in the current picture.

The encoding apparatus constructs MVP candidate lists for the currentblock based on neighboring blocks of the current block S1810. The MVPcandidate lists may include MVP candidate list L0 and MVP candidate listL1. In one example, the neighboring blocks may be included in a currentpicture including the current block. In another example, the neighboringblocks may be included in the previous (reference) picture or the next(reference) picture of the current picture. Here, the POC of theprevious picture may be smaller than the POC of the current picture, andthe POC of a subsequent picture may be larger than the POC of thecurrent picture. According to one example, a POC difference between thecurrent picture and the previous (reference) picture of the currentpicture may be larger than zero. In another example, a POC differencebetween the current picture and the next (reference) picture of thecurrent picture may be smaller than zero. However, the description aboveis only an example.

The encoding apparatus may derive MVPs for the current block based onthe MVP candidate lists S1820. MVPs may include MVPL0 and MVPL1. MVPL0may be derived from the MVP candidate list L0, and MVPL1 may be derivedfrom the MVP candidate list L1. The encoding apparatus may derive theoptimal motion vector predictor candidate from among the motion vectorpredictor candidates included in the MVP candidate list. The encodingapparatus may generate selection information (for example, an MVP flagor an MVP index) indicating the optimal motion vector predictorcandidate.

The encoding apparatus generates prediction related information thatincludes the inter prediction mode S1830. In one example, the predictionrelated information may include information on the motion vectordifferences (MVDs) for the current block. The prediction relatedinformation may include information on MMVD and information on SMVD.

The encoding apparatus derives motion information for prediction of thecurrent block based on the information on the MVPs and the MVDs S1840.For example, the motion information may include reference indices(symmetric motion vector difference reference indices) for SMVD. Thereference indices for SMVD may indicate reference pictures for the SMVDapplication. The reference indices for SMVD may include reference indexL0 (RefIdxSumL0) and reference index L1 (RefIdxSumL1).

The encoding apparatus generates prediction samples based on the motioninformation S1850. The encoding apparatus may generate the predictionsamples based on the motion vectors and the reference picture indicesincluded in the motion information. For example, the prediction samplesmay be generated based on the blocks (or samples) indicated by themotion vector among the blocks (or samples) within the referencepictures indicated by the reference picture indices.

The encoding apparatus derives residual information based on theprediction samples S1860. Specifically, the encoding apparatus mayderive residual samples based on the prediction samples and the originalsamples. The encoding apparatus may derive residual information based onthe residual samples. The transform and quantization processes may beperformed to derive the residual information.

The encoding apparatus encodes image/video information including theprediction related information and the residual information S1870. Theencoded image/video information may be output in the form of abitstream. The bitstream may be transmitted to the decoding apparatusthrough a network or a (digital) storage medium.

The image/video information may include various types of informationaccording to an embodiment of the present disclosure. For example, theimage/video information may include information disclosed in at leastone of Tables 1 to 34 described above.

In one embodiment, the motion information may include motion vectors(MVs) and symmetric motion vector difference reference indices. The MVsmay include MVL0 for L0 prediction and MVL1 for L1 prediction. Thesymmetric motion vector difference reference indices may includesymmetric motion vector difference reference index L0 for L0 predictionand symmetric motion vector difference reference index L1 for L1prediction. The information on the MVDs may include information on theMVDL0 for L0 prediction. In one example, the information on the MVDL1for L1 prediction may be derived based on the information on the MVDL0.In another example, information on MVDL0 and/or information on MVDL1 maybe derived based on neighboring blocks for prediction of the currentblock. The encoding apparatus may exclude the information on MVDL1 whenencoding image/video information. The MVL0 may be derived based on theinformation on the MVDL0, and the MVL1 may be derived based on theinformation on the MVDL1. The symmetric motion vector differencereference index L0 and the symmetric motion vector difference referenceindex L1 may be derived based on short-term reference pictures amongreference pictures included in reference picture lists.

In one embodiment, the MVPs may include the MVPL0 for L0 prediction andthe MVPL1 for L1 prediction. The MVL0 may be derived based on a sum ofthe MVDL0 and the MVPL0. The MVL1 may be derived based on a sum of theMVDL1 and the MVPL1.

In one embodiment, the prediction related information may includeinformation on symmetric motion vector differences (information on SMVDor SMVD flag information). When the symmetric motion vector differencereference index L0 and the symmetric motion vector difference referenceindex L1 are derived based on picture order count (POC) differencesbetween the short-term reference pictures and the current pictureincluding the current block, the value of the information on thesymmetric motion vector differences may be 1.

In one embodiment, the size of the MVDL1 may be the same as that of theMVDL0. The sign of the MVDL1 may be opposite to that of the MVDL0.

In one embodiment, the short-term reference pictures may includeshort-term reference picture L0 and short-term reference picture L1. Forexample, the symmetric motion vector difference reference index L0 mayindicate the short-term reference picture L0. Also, the symmetric motionvector difference reference index L1 may indicate the short-termreference picture L1.

In one embodiment, the reference picture lists may include referencepicture list 0. The reference picture list 0 may include the short-termpicture L0. the symmetric motion vector difference reference index L0may be derived based on the picture order count (POC) differencesbetween each of the short-term reference pictures included in thereference picture list 0 and the current picture including the currentblock.

In one embodiment, based on the comparison between the POC differences,the symmetric motion vector difference reference index L0 may bederived.

In one embodiment, the reference picture list 0 may further includeanother short-term reference picture L0. The POC differences may includea first POC difference between the short-term reference picture 0 andthe current picture and a second POC difference between anothershort-term reference picture L0 and the current picture. The first POCdifference may be smaller than the second POC difference.

FIGS. 20 and 21 illustrate an image/video decoding method and oneexample of a related component according to an embodiment(s) of thepresent disclosure. The decoding apparatus of FIG. 3 may perform themethod of FIG. 20. Specifically, for example, the entropy decoder 310 ofthe decoding apparatus may perform the S2000 step of FIG. 20, and thepredictor 330 of the decoding apparatus may perform the S2010 to S2050steps. The method of FIG. 20 may include the embodiments of the presentdisclosure described above.

Referring to FIG. 20, the decoding apparatus receives/obtainsimage/video information S2000. The decoding apparatus may receive/obtainthe image/video information through a bitstream. The image/videoinformation may include prediction related information (includingprediction mode information), information on MVDs, and/or residualinformation. The prediction related information may include informationon MMVD and information on SMVD. Also, the image/video information mayinclude various types of information according to an embodiment of thepresent disclosure. For example, the image/video information may includethe information described with reference to FIGS. 1 to 17 and/or theinformation disclosed in at least one of Tables 1 to 34 described above.

The decoding apparatus derives the inter prediction mode for a currentblock based on the prediction related information S2010. Here, the interprediction mode may include the merge mode, AMVP mode (the mode usingmotion vector predictor candidates), MMVD, and SMVD.

The decoding apparatus constructs MVP candidate lists for a currentblock based on neighboring blocks of the current block S2020. The MVPcandidate lists may include MVP candidate list L0 and MVP candidate listL1. In one example, the neighboring blocks may be included in thecurrent picture that includes the current block. In another example, theneighboring blocks may be included in the previous (reference) pictureof the next (reference) picture of the current picture. Here, the POC ofthe previous picture may be smaller than the POC of the current picture,and the POC of a subsequent picture may be larger than the POC of thecurrent picture. According to one example, the POC difference betweenthe current picture and the previous (reference) picture of the currentpicture may be larger than 0. In another example, the POC differencebetween the current picture and the next (reference) picture of thecurrent picture may be smaller than 0. However, the description above isonly an example.

The decoding apparatus may derive MVPs for the current block based onthe MVP candidate lists S2030. The MVPs may include MVPL0 and MVPL1.MVPL0 may be derived from MVP candidate list L0, and MVPL1 may bederived from MVP candidate list L1. The decoding apparatus may derivethe optimal motion vector predictor candidate from among motion vectorpredictor candidates included in the MVP candidate list. The encodingapparatus may generate selection information (for example, MVP flag orMVP index) indicating the optimal motion vector predictor candidate.

The decoding apparatus derives motion information for the current blockbased on the information related to the MVDs and the MVPs S2040. Forexample, the motion information may include reference indices for SMVD.The reference indices for SMVD may indicate reference pictures for theSMVD application. The reference indices for SMVD may include referenceindex L0 (RefIdxSumL0) and reference index L1 (RefIdxSumL1).

The decoding apparatus generates prediction samples based on the motioninformation S2050. The decoding apparatus may generate the predictionsamples based on the motion vectors and the reference picture indicesincluded in the motion information. For example, the prediction samplesmay be generated based on the blocks (or samples) indicated by themotion vector from among the blocks (or samples) within the referencepictures indicated by the reference picture indices.

The decoding apparatus may generate residual samples based on theresidual information. Specifically, the decoding apparatus may derivequantized transform coefficients based on the residual information. Thequantized transform coefficients may have a one-dimensional vector formbased on the coefficient scan order. The decoding apparatus may derivetransform coefficients based on the dequantization procedure for thequantized transform coefficients. The decoding apparatus may deriveresidual samples based on the inverse transform procedure for thetransform coefficients.

The decoding apparatus may generate reconstructed samples of the currentpicture based on the prediction samples and the residual samples. Thedecoding apparatus may further perform a filtering procedure to generate(modified) reconstructed samples.

In one embodiment, the motion information may include motion vectors(MVs) and symmetric motion vector difference reference indices. The MVsmay include MVL0 for L0 prediction and MVL1 for L1 prediction. Thesymmetric motion vector difference reference indices may includesymmetric motion vector difference reference index L0 for L0 predictionand symmetric motion vector difference reference index L1 for L1prediction. The information on the MVDs may include information on theMVDL0 for L0 prediction. In one example, the information on the MVDL1for L1 prediction may be derived based on the information on the MVDL0.In another example, information on MVDL0 and/or information on MVDL1 maybe derived based on neighboring blocks for prediction of the currentblock. The encoding apparatus may exclude the information on MVDL1 whenencoding image/video information. The MVL0 may be derived based on theinformation on the MVDL0, and the MVL1 may be derived based on theinformation on the MVDL1. The symmetric motion vector differencereference index L0 and the symmetric motion vector difference referenceindex L1 may be derived based on short-term reference pictures amongreference pictures included in reference picture lists.

In one embodiment, the MVPs may include the MVPL0 for L0 prediction andMVPL1 for L1 prediction. The MVL0 may be derived based on a sum of theMVDL0 and the MVPL0. The MVL1 may be derived based on a sum of the MVDL1and the MVPL1.

In one embodiment, the prediction related information may includeinformation on symmetric motion vector differences (information on SMVDor SMVD flag information). For example, when the value of information onthe symmetric motion vector differences is 1, the symmetric motionvector difference reference index L0 and the symmetric motion vectordifference reference index L1 may be derived based on POC differencesbetween the short-term reference pictures and the current pictureincluding the current block.

In one embodiment, the size of the MVDL1 may be the same as that of theMVDL0. The sign of the MVDL1 may be opposite to that of the MVDL0.

In one embodiment, the short-term reference pictures may includeshort-term reference picture L0 and short-term reference picture L1. Forexample, the symmetric motion vector difference reference index L0 mayindicate the short-term reference picture L0. Also, the symmetric motionvector difference reference index L1 may indicate the short-termreference picture L1.

In one embodiment, the reference picture lists may include referencepicture list 0. The reference picture list 0 may include the short-termpicture L0. the symmetric motion vector difference reference index L0may be derived based on the picture order count (POC) differencesbetween each of the short-term reference pictures included in thereference picture list 0 and the current picture including the currentblock.

In one embodiment, based on the comparison between the POC differences,the symmetric motion vector difference reference index L0 may bederived.

In one embodiment, the reference picture list 0 may further includeanother short-term reference picture L0. The POC differences may includea first POC difference between the short-term reference picture 0 andthe current picture and a second POC difference between anothershort-term reference picture L0 and the current picture. The first POCdifference may be smaller than the second POC difference.

In one embodiment, the symmetric motion vector difference referenceindex L0 and the symmetric motion vector difference reference index L1(for example, ref_idx_l1[x0][y0], ref_idx_l1[x][y0]) may not be signaleddirectly but may be derived based on the information on the symmetricmotion vector differences (for example, sym_mvd_flag).

In the above-described embodiment, the methods are described based onthe flowchart having a series of steps or blocks. The present disclosureis not limited to the order of the above steps or blocks. Some steps orblocks may occur simultaneously or in a different order from other stepsor blocks as described above. Further, those skilled in the art willunderstand that the steps shown in the above flowchart are notexclusive, that further steps may be included, or that one or more stepsin the flowchart may be deleted without affecting the scope of thepresent disclosure.

The method according to the above-described embodiments of the presentdocument may be implemented in software form, and the encoding deviceand/or decoding device according to the present document is, forexample, may be included in the device that performs the imageprocessing of a TV, a computer, a smart phone, a set-top box, a displaydevice, etc.

When the embodiments in the present document are implemented insoftware, the above-described method may be implemented as a module(process, function, etc.) that performs the above-described function. Amodule may be stored in a memory and executed by a processor. The memorymay be internal or external to the processor, and may be coupled to theprocessor by various well-known means. The processor may include anapplication-specific integrated circuit (ASIC), other chipsets, logiccircuits, and/or data processing devices. Memory may include read-onlymemory (ROM), random access memory (RAM), flash memory, memory cards,storage media, and/or other storage devices. That is, the embodimentsdescribed in the present document may be implemented and performed on aprocessor, a microprocessor, a controller, or a chip. For example, thefunctional units shown in each figure may be implemented and performedon a computer, a processor, a microprocessor, a controller, or a chip.In this case, information on instructions or an algorithm forimplementation may be stored in a digital storage medium.

In addition, the decoding apparatus and the encoding apparatus to whichthe present disclosure is applied may be included in a multimediabroadcasting transmission/reception apparatus, a mobile communicationterminal, a home cinema video apparatus, a digital cinema videoapparatus, a surveillance camera, a video chatting apparatus, areal-time communication apparatus such as video communication, a mobilestreaming apparatus, a storage medium, a camcorder, a VoD serviceproviding apparatus, an Over the top (OTT) video apparatus, an Internetstreaming service providing apparatus, a three-dimensional (3D) videoapparatus, a teleconference video apparatus, a transportation userequipment (i.e., vehicle user equipment, an airplane user equipment, aship user equipment, etc.) and a medical video apparatus and may be usedto process video signals and data signals. For example, the Over the top(OTT) video apparatus may include a game console, a blue-ray player, aninternet access TV, a home theater system, a smart phone, a tablet PC, aDigital Video Recorder (DVR), and the like.

Furthermore, the processing method to which the present document isapplied may be produced in the form of a program that is to be executedby a computer and may be stored in a computer-readable recording medium.Multimedia data having a data structure according to the presentdisclosure may also be stored in computer-readable recording media. Thecomputer-readable recording media include all types of storage devicesin which data readable by a computer system is stored. Thecomputer-readable recording media may include a BD, a Universal SerialBus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, a magnetic tape, afloppy disk, and an optical data storage device, for example.Furthermore, the computer-readable recording media includes mediaimplemented in the form of carrier waves (i.e., transmission through theInternet). In addition, a bitstream generated by the encoding method maybe stored in a computer-readable recording medium or may be transmittedover wired/wireless communication networks.

In addition, the embodiments of the present document may be implementedwith a computer program product according to program codes, and theprogram codes may be performed in a computer by the embodiments of thepresent document. The program codes may be stored on a carrier which isreadable by a computer.

FIG. 22 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

Referring to FIG. 22, the content streaming system to which theembodiment(s) of the present document is applied may largely include anencoding server, a streaming server, a web server, a media storage, auser device, and a multimedia input device.

The encoding server compresses content input from multimedia inputdevices such as a smartphone, a camera, a camcorder, etc. Into digitaldata to generate a bitstream and transmit the bitstream to the streamingserver. As another example, when the multimedia input devices such assmartphones, cameras, camcorders, etc. directly generate a bitstream,the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstreamgenerating method to which the embodiment(s) of the present disclosureis applied, and the streaming server may temporarily store the bitstreamin the process of transmitting or receiving the bitstream.

The streaming server transmits the multimedia data to the user devicebased on a user's request through the web server, and the web serverserves as a medium for informing the user of a service. When the userrequests a desired service from the web server, the web server deliversit to a streaming server, and the streaming server transmits multimediadata to the user. In this case, the content streaming system may includea separate control server. In this case, the control server serves tocontrol a command/response between devices in the content streamingsystem.

The streaming server may receive content from a media storage and/or anencoding server. For example, when the content is received from theencoding server, the content may be received in real time. In this case,in order to provide a smooth streaming service, the streaming server maystore the bitstream for a predetermined time.

Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcasting terminal, a personal digitalassistant (PDA), a portable multimedia player (PMP), navigation, a slatePC, tablet PCs, ultrabooks, wearable devices (ex. Smartwatches, smartglasses, head mounted displays), digital TVs, desktops computer, digitalsignage, and the like. Each server in the content streaming system maybe operated as a distributed server, in which case data received fromeach server may be distributed.

Each server in the content streaming system may be operated as adistributed server, and in this case, data received from each server maybe distributed and processed.

The claims described herein may be combined in various ways. Forexample, the technical features of the method claims of the presentdocument may be combined and implemented as an apparatus, and thetechnical features of the apparatus claims of the present document maybe combined and implemented as a method. In addition, the technicalfeatures of the method claim of the present document and the technicalfeatures of the apparatus claim may be combined to be implemented as anapparatus, and the technical features of the method claim of the presentdocument and the technical features of the apparatus claim may becombined and implemented as a method.

What is claimed is:
 1. An image decoding method performed by a decodingapparatus, the method comprising: receiving image information includingprediction related information and information on motion vectordifferences (MVDs) through a bitstream; deriving an inter predictionmode for a current block based on the prediction related information;constructing motion vector predictor (MVP) candidate lists for thecurrent block based on neighboring blocks of the current block; derivingMVPs for the current block based on the MVP candidate lists; derivingmotion information for the current block based on the information on theMVDs and the MVPs; and generating predicted samples for the currentblock based on the motion information, wherein bi-prediction is appliedto the current block, wherein the motion information includes motionvectors (MVs) and symmetric motion vector difference reference indices,wherein the MVs include an MVL0 for L0 prediction and an MVL1 for L1prediction, wherein the symmetric motion vector difference referenceindices include a symmetric motion vector difference reference index L0for the L0 prediction and a symmetric motion vector difference referenceindex L1 for the L1 prediction, wherein the information on the MVDsincludes information on an MVDL0 for the L0 prediction, whereininformation on an MVDL1 for the L1 prediction is derived based on theinformation on the MVDL0, wherein the MVDL0 is derived based on theinformation on the MVDL0, wherein the MVDL1 is derived based on theinformation on the MVDL1, wherein the symmetric motion vector differencereference index L0 and the symmetric motion vector difference referenceindex L1 are derived based on short-term reference pictures amongreference pictures included in reference picture lists.
 2. The method ofclaim 1, wherein the MVPs include the MVPL0 for L0 prediction and theMVPL1 for L1 prediction, the MVL0 is derived based on a sum of the MVDL0and the MVPL0, and the MVL1 is derived based on a sum of the MVDL1 andthe MVPL1.
 3. The method of claim 1, wherein the prediction relatedinformation includes information on symmetric motion vector differences,and when the value of the information on symmetric motion vectordifferences is 1, the symmetric motion vector difference reference indexL0 and the symmetric motion vector difference reference index L1 arederived based on picture order count (POC) differences between theshort-term reference pictures and a current picture including thecurrent block.
 4. The method of claim 1, wherein the size of the MVDL1is the same as that of the MVDL0, and the sign of the MVDL1 is oppositeto that of the MVDL0.
 5. The method of claim 1, wherein the short-termreference pictures include short-term reference picture L0 andshort-term reference picture L1, the symmetric motion vector differencereference index L0 indicates the short-term reference picture L0, andthe symmetric motion vector difference reference index L1 indicates theshort-term reference picture L1.
 6. The method of claim 5, wherein thereference picture lists include reference picture list 0, the referencepicture list 0 includes the short-term reference picture L0, and thesymmetric motion vector difference reference index L0 is derived basedon picture order count (POC) differences between each of short-termreference pictures included in the reference picture list 0 and thecurrent picture including the current block.
 7. The method of claim 6,wherein the symmetric motion vector difference reference index L0 isderived based on comparison between the POC differences.
 8. The methodof claim 7, wherein the reference picture list 0 further includesanother short-term reference picture L0, the POC differences include afirst POC difference between the short-term reference picture L0 and thecurrent picture and a second POC difference between another short-termreference picture L0 and the current picture, and the first POCdifference is smaller than the second POC difference.
 9. The method ofclaim 5, wherein the prediction related information includes informationon symmetric motion vector differences, and the symmetric motion vectordifference reference index L0 and the symmetric motion vector differencereference index L1 are not signaled directly but are derived based oninformation on the symmetric motion vector differences.
 10. An imageencoding method performed by an encoding apparatus, the methodcomprising: deriving an inter prediction mode for a current block;constructing motion vector predictor (MVP) candidate lists for thecurrent block based on neighboring blocks of the current block; derivingMVPs for the current block based on the MVP candidate lists; generatingprediction related information including information on the interprediction mode and motion vector differences (MVDs) for the currentblock; deriving motion information for the current block based on theinformation on the MVPs and the MVDs; generating predicted samples forthe current block based on the motion information; generating residualinformation based on the predicted samples; and encoding imageinformation including the prediction related information and theresidual information, wherein the motion information includes motionvectors (MVs) and symmetric motion vector difference reference indices,wherein the MVs include MVL0 for L0 prediction and MVL1 for L1prediction, wherein the symmetric motion vector difference referenceindices include the symmetric motion vector difference reference indexL0 for L0 prediction and the symmetric motion vector differencereference index L1 for L1 prediction, wherein the information on theMVDs includes the information on the MVDL0 for L0 prediction, whereinthe information on the MVDL1 for L1 prediction is derived based on theinformation on the MVDL0, wherein the MVL0 is derived based on theinformation on the MVDL0, wherein the MVL1 is derived based on theinformation on the MVDL1, wherein the symmetric motion vector differencereference index L0 and the symmetric motion vector difference referenceindex L1 are derived based on short-term reference pictures amongreference pictures included in reference picture lists.
 11. The methodof claim 10, wherein the MVPs include the MVPL0 for L0 prediction andthe MVPL1 for L1 prediction, the MVL0 is derived based on a sum of theMVDL0 and the MVPL0, and the MVL1 is derived based on a sum of the MVDL1and the MVPL1.
 12. The method of claim 10, wherein the predictionrelated information includes information on symmetric motion vectordifferences, and when the symmetric motion vector difference referenceindex L0 and the symmetric motion vector difference reference index L1are derived based on picture order count (POC) differences between theshort-term reference pictures and a current picture including thecurrent block, the value of the information on the symmetric motionvector differences is
 1. 13. The method of claim 10, wherein the size ofthe MVDL1 is the same as that of the MVDL0, and the sign of the MVDL1 isopposite to that of the MVDL0.
 14. The method of claim 10, wherein theshort-term reference pictures include short-term reference picture L0and short-term reference picture L1, the symmetric motion vectordifference reference index L0 indicates the short-term reference pictureL0, and the symmetric motion vector difference reference index L1indicates the short-term reference picture L1.
 15. The method of claim14, wherein the reference picture lists include reference picture list0, the reference picture list 0 includes the short-term referencepicture L0, and the symmetric motion vector difference reference indexL0 is derived based on picture order count (POC) differences betweeneach of short-term reference pictures included in the reference picturelist 0 and a current picture including the current block.
 16. The methodof claim 15, wherein the symmetric motion vector difference referenceindex L0 is derived based on comparison between the POC differences. 17.The method of claim 16, wherein the reference picture list 0 furtherincludes another short-term reference picture L0, the POC differencesinclude a first POC difference between the short-term reference pictureL0 and the current picture and a second POC difference between anothershort-term reference picture L0 and the current picture, and the firstPOC difference is smaller than the second POC difference.
 18. Anon-transitory computer readable storage medium storing a bitstreamgenerated by a method, wherein the method comprising: deriving an interprediction mode for a current block; constructing motion vectorpredictor (MVP) candidate lists for the current block based onneighboring blocks of the current block; deriving MVPs for the currentblock based on the MVP candidate lists; generating prediction relatedinformation including information on the inter prediction mode andmotion vector differences (MVDs) for the current block; deriving motioninformation for the current block based on the information on the MVPsand the MVDs; generating predicted samples for the current block basedon the motion information; generating residual information based on thepredicted samples; and encoding image information including theprediction related information and the residual information to outputthe bitstream, wherein the motion information includes motion vectors(MVs) and symmetric motion vector difference reference indices, whereinthe MVs include MVL0 for L0 prediction and MVL1 for L1 prediction,wherein the symmetric motion vector difference reference indices includethe symmetric motion vector difference reference index L0 for L0prediction and the symmetric motion vector difference reference index L1for L1 prediction, wherein the information on the MVDs includes theinformation on the MVDL0 for L0 prediction, wherein the information onthe MVDL1 for L1 prediction is derived based on the information on theMVDL0, wherein the MVL0 is derived based on the information on theMVDL0, wherein the MVL1 is derived based on the information on theMVDL1, wherein the symmetric motion vector difference reference index L0and the symmetric motion vector difference reference index L1 arederived based on short-term reference pictures among reference picturesincluded in reference picture lists.